-- Logs begin at Tue 2022-07-19 10:18:56 UTC, end at Tue 2022-07-19 11:21:30 UTC. -- Jul 19 10:55:33.799439 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Jul 19 10:55:33.806717 dv-cinder-master-3-iscsi devstack@n-api.service[103327]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Jul 19 10:55:33.810050 dv-cinder-master-3-iscsi devstack@n-api.service[103327]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jul 19 10:55:33 2022] *** Jul 19 10:55:33.810050 dv-cinder-master-3-iscsi devstack@n-api.service[103327]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 19 10:55:33.810050 dv-cinder-master-3-iscsi devstack@n-api.service[103327]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 19 10:55:33.810050 dv-cinder-master-3-iscsi devstack@n-api.service[103327]: nodename: dv-cinder-master-3-iscsi Jul 19 10:55:33.810050 dv-cinder-master-3-iscsi devstack@n-api.service[103327]: machine: x86_64 Jul 19 10:55:33.810050 dv-cinder-master-3-iscsi devstack@n-api.service[103327]: clock source: unix Jul 19 10:55:33.810050 dv-cinder-master-3-iscsi devstack@n-api.service[103327]: pcre jit disabled Jul 19 10:55:33.810050 dv-cinder-master-3-iscsi devstack@n-api.service[103327]: detected number of CPU cores: 4 Jul 19 10:55:33.810050 dv-cinder-master-3-iscsi devstack@n-api.service[103327]: current working directory: / Jul 19 10:55:33.810050 dv-cinder-master-3-iscsi devstack@n-api.service[103327]: detected binary path: /usr/bin/uwsgi-core Jul 19 10:55:33.810050 dv-cinder-master-3-iscsi devstack@n-api.service[103327]: your processes number limit is 47680 Jul 19 10:55:33.810050 dv-cinder-master-3-iscsi devstack@n-api.service[103327]: your memory page size is 4096 bytes Jul 19 10:55:33.810050 dv-cinder-master-3-iscsi devstack@n-api.service[103327]: detected max file descriptor number: 2048 Jul 19 10:55:33.810050 dv-cinder-master-3-iscsi devstack@n-api.service[103327]: lock engine: pthread robust mutexes Jul 19 10:55:33.810050 dv-cinder-master-3-iscsi devstack@n-api.service[103327]: thunder lock: enabled Jul 19 10:55:33.810869 dv-cinder-master-3-iscsi devstack@n-api.service[103327]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Jul 19 10:55:33.810869 dv-cinder-master-3-iscsi devstack@n-api.service[103327]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 19 10:55:33.853540 dv-cinder-master-3-iscsi devstack@n-api.service[103327]: Python main interpreter initialized at 0x558002c8d9c0 Jul 19 10:55:33.853540 dv-cinder-master-3-iscsi devstack@n-api.service[103327]: python threads support enabled Jul 19 10:55:33.853540 dv-cinder-master-3-iscsi devstack@n-api.service[103327]: your server socket listen backlog is limited to 100 connections Jul 19 10:55:33.853540 dv-cinder-master-3-iscsi devstack@n-api.service[103327]: your mercy for graceful operations on workers is 90 seconds Jul 19 10:55:33.853898 dv-cinder-master-3-iscsi devstack@n-api.service[103327]: mapped 403077 bytes (393 KB) for 2 cores Jul 19 10:55:33.853975 dv-cinder-master-3-iscsi devstack@n-api.service[103327]: *** Operational MODE: preforking *** Jul 19 10:55:33.854203 dv-cinder-master-3-iscsi devstack@n-api.service[103327]: *** uWSGI is running in multiple interpreter mode *** Jul 19 10:55:33.854203 dv-cinder-master-3-iscsi devstack@n-api.service[103327]: spawned uWSGI master process (pid: 103327) Jul 19 10:55:33.854394 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Jul 19 10:55:33.855778 dv-cinder-master-3-iscsi devstack@n-api.service[103327]: spawned uWSGI worker 1 (pid: 103328, cores: 1) Jul 19 10:55:33.855778 dv-cinder-master-3-iscsi devstack@n-api.service[103327]: spawned uWSGI worker 2 (pid: 103329, cores: 1) Jul 19 10:55:33.855778 dv-cinder-master-3-iscsi devstack@n-api.service[103327]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 19 10:55:34.509469 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Jul 19 10:55:34.518185 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Jul 19 10:55:35.629360 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 19 10:55:35.629360 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: from cryptography.utils import int_from_bytes Jul 19 10:55:35.629984 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 19 10:55:35.629984 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: from cryptography.utils import int_from_bytes Jul 19 10:55:35.650820 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 19 10:55:35.650820 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: from cryptography.utils import int_from_bytes Jul 19 10:55:35.651417 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 19 10:55:35.651417 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: from cryptography.utils import int_from_bytes Jul 19 10:55:38.102331 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 19 10:55:38.102885 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 19 10:55:38.103104 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 19 10:55:38.103467 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 19 10:55:38.103728 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 19 10:55:38.104119 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.104445 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.104797 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.105122 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.105455 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.105638 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 19 10:55:38.105996 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.106084 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 19 10:55:38.106169 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.106392 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.106486 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 19 10:55:38.106729 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.106864 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 19 10:55:38.106988 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 19 10:55:38.107128 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.107305 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.107391 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.107634 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.107744 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.107960 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.108064 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.108325 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.108920 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.109155 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.109251 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.109475 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.109562 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.109784 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.109866 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.110109 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.110199 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.110418 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.110502 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.110721 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.110913 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.111035 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.111361 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.111446 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.111668 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.111751 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.111971 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.112062 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.112279 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.112381 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.112593 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.112682 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.112920 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.113016 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.113256 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.113401 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.113567 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.113714 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.113892 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.114202 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.114296 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.114509 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.114618 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.114815 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.114937 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.115140 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.115264 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.115461 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.115545 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.115772 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.115953 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.116337 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.116531 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.116662 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.116871 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.117031 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.117194 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.117305 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.117519 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.117612 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.117831 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.117991 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.118161 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.118305 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.118484 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.118615 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.118809 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.118927 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.119137 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.119339 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.119437 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.119650 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.119750 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.119959 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.120147 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.120281 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.120377 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.120610 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.120713 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.120942 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.121037 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.121206 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.121299 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.121517 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.121601 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.121843 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.121928 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.122169 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.122260 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.122472 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.122562 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.122737 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.122825 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.122992 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.123077 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.123318 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.123416 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.123642 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.123735 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.123979 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.124060 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.124231 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.124327 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.124535 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.124619 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.124879 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.124976 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.125200 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.125292 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.125508 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.125593 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.125764 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.125860 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.126016 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.126101 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.126332 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.126416 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.126650 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.126734 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.126962 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.127060 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.127268 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.127367 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.127571 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.127676 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.127873 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.128023 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.33 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.128163 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.128279 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.33 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.128463 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.128589 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.128797 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.128957 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.129104 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.129283 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.129424 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.129592 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.129733 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.129914 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.130053 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.130238 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.130358 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.130549 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.130716 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.130856 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.131055 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.33 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.131165 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.131307 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.33 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.131487 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.131611 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.131820 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.131946 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.132128 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.132263 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.132382 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.132570 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.132683 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.132920 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.133008 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.133240 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.133325 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.133547 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.133631 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.133850 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.133933 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.134153 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.134238 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.134470 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.134555 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.134781 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.134863 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.135088 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.135181 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.135334 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.135429 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.135637 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.135738 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.135938 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.136029 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.136253 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.136348 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.136556 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.136649 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.136884 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.136984 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.137189 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.137275 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.137484 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.137575 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.137785 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.137877 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.138087 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.138177 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.138390 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.138486 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.138691 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.138781 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.138993 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.139081 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.139293 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.139440 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.139623 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.139706 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.139928 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.140010 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.140228 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.140321 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.140541 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.140624 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.140871 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.140967 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.141176 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.141260 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.141480 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.141563 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.141783 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.141864 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.142084 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.142176 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.142388 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.142471 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.142688 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.142769 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.142991 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.143083 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.143297 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.143379 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.143598 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.143687 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.143898 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.143981 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.144201 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.144284 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.144504 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.144585 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.144828 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.144929 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.145132 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.145215 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.145433 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.145515 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.145731 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.145824 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.146043 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.146125 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.146347 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.146507 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.146648 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.146732 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.146946 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.147031 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.147247 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.147343 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.147547 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.147631 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.147850 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.147932 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.148150 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.148246 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.148450 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.148541 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.148767 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.148853 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.149085 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.149181 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.149402 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.149489 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.149707 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.149790 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.150011 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.150102 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.150313 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.150403 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.150629 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 10:55:38.150713 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.150959 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.151043 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.151288 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.151373 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.151611 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.151701 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.151928 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.152010 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.152244 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.152325 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.152574 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.152667 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.152912 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.152996 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.153223 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.153317 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.153535 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.153636 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.153848 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.153962 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.154158 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.154242 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.154469 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.154561 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.154779 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.154863 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.155617 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.155744 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.155940 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.156036 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.156270 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.156354 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.156585 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.156677 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.156925 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.157025 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.157242 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.157370 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.157551 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.157641 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.157863 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.157955 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.158174 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.158271 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.158494 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.158581 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.158804 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.158897 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.159122 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.159204 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.159443 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.159533 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.159759 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.159847 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.160072 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.160155 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.160385 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.160475 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.160728 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.160816 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.161025 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.161108 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.161335 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.161430 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.161652 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.161737 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.161966 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.162050 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.162276 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.162358 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.162591 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.162676 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.162900 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.163032 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.163209 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.163292 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.163516 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.163605 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.163821 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.163903 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.164127 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.164224 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.164434 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.164518 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.164765 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.164852 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.165080 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.165176 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.165398 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.165481 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.165715 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.165844 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.166024 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.166123 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.166332 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.166423 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.166640 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.166730 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.166943 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.167038 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.167245 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.167330 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.167549 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.167637 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.167851 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.167935 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.168164 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.168250 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.168476 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.168596 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.168807 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.168948 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.169135 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.169234 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.169449 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.169533 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.169755 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.169842 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.170076 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.170158 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.170380 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.170469 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.170708 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.171306 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.171539 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.171630 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.171854 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.171947 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.172166 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.172257 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.172480 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.172571 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.172817 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.172922 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.173127 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.173210 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.173433 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.173518 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.173746 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.173837 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.174048 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.174137 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.174352 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.174436 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.174666 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.174758 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.174974 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.175065 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.175283 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.175373 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.175594 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.175679 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.175901 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.175985 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.176215 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.176296 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.176522 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.176608 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.176862 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.176984 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.177174 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.177273 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.177485 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.177571 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.177789 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.177881 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.178094 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.178178 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.178400 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.178485 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.178704 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.178788 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.179008 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.179098 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.179317 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.179400 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.179620 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.179705 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.179923 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.180017 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.180224 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.180306 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.180525 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.180607 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.180854 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.180952 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.181159 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.181245 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.181462 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.181546 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.181765 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.181853 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.182067 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.182158 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.182386 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.182469 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.182695 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.182786 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.182997 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.183080 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.183343 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.183441 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.183599 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.183685 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.183937 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.184023 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.184240 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.184324 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.184557 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.184658 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.184899 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.184985 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.185220 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.185303 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.185533 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.185625 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.185841 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.185931 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.186155 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.186252 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.186500 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.186584 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.186808 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.186890 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.187120 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.187212 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.187430 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.187514 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.187748 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.187831 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.188056 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.188154 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.188373 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.188460 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.188681 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.188820 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.189022 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.189118 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.189327 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.189410 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.189638 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.189722 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.189939 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.190030 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.190250 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.190338 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.190569 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.190652 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.190880 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.190965 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.191201 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.191284 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.191513 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.191596 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.191821 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.191905 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.192134 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.192224 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.192441 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.192526 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.192766 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.192853 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.193075 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.193169 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.193384 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.193467 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.193700 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.193786 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.194008 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.194092 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.194321 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.194450 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.194625 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.194716 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.194946 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.195029 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.195251 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.195346 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.195562 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.195652 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.195861 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.195951 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.196166 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.196299 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.196478 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.196562 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.196818 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.196903 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.197125 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.197218 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.197435 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.197520 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.197744 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.197839 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.198054 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.198137 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.198361 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.198450 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.198666 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.198748 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.198965 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.199051 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.199265 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.199355 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.199571 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.199656 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.199880 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.199963 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.200185 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.200268 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.200491 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.200576 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.200823 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.200911 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.201131 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.201215 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.201434 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.201528 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.201736 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.201820 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.202062 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.202146 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.202365 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.202456 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.202670 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.202755 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.202977 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.203060 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.203296 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.203388 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.203597 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.203687 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.203905 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.203987 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.204207 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.204299 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.204519 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.204601 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.204850 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.204955 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.205155 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.205248 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.205457 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.205540 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.205757 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.205846 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.206075 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.206159 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.206386 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.206469 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.206685 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.206775 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.206992 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.207076 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.207301 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.207391 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.207605 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.207687 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.207915 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.208000 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.208220 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.208310 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.208523 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.208606 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.208856 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.208944 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.209163 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.209247 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.209467 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.209559 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.209775 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.209860 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.210084 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.210339 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.210423 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.210676 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.210762 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.210995 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.211078 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.211301 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.211390 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.211611 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.211696 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.211919 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.212010 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.212224 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.212321 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.212533 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.212618 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.212871 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.212956 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.213211 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.213428 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.213513 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.213759 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.213844 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.214074 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.214164 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.214391 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.214487 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.214701 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.214784 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.215015 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.215099 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.215322 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.215417 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.215636 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.215721 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.215944 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.216029 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.216250 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.216388 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.216555 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.216640 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.216886 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.216972 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.217201 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.217283 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.217505 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.218078 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.218323 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.218422 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.218627 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.218709 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.218931 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.219014 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.219240 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.219328 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.219543 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.219626 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.219846 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.219938 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.220147 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.220229 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.220448 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.220540 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.220791 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.220902 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.221098 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.221197 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.221406 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.221507 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.221709 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.221801 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.222030 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.222122 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.222336 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.222437 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.222641 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.222733 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.222941 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.223033 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.223244 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.223338 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.223541 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.223631 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.223849 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.223934 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.224156 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.224248 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.224457 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.224543 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.224799 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.224887 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.225111 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.225197 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.225423 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.225507 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.225731 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.225814 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.226033 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.226118 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.226338 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.226470 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.226642 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.226733 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.226945 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.227035 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.227249 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.227333 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.227548 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.227642 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.227846 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.227937 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.228148 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.228274 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.228452 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.228551 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.228774 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.228862 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.229075 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.229158 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.229375 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.229469 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.229673 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.229756 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.229979 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.230063 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.230282 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.230366 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.230580 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.230670 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.230880 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.230969 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.231185 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.231269 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.231486 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.231577 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.231781 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.231872 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.232088 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.232173 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.232386 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.232469 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.232685 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.232804 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.233011 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.233096 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.233867 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.234026 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.234160 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.234258 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.234466 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.234564 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.234765 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.234848 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.235064 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.235161 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.235383 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.235473 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.235686 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.235769 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.235987 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.236077 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.236286 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.236375 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.236585 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.236675 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.236910 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.237010 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.237210 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.237295 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.237510 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.237592 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.237812 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.237906 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.238115 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.238207 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.238412 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.238495 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.238717 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.238802 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.239016 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.239107 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.239316 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.239414 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.239616 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.239702 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.239914 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.239996 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.240209 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.240298 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.240507 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.240594 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.240828 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.240926 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.241392 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.241500 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.241691 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.241776 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.242010 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.242094 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.242380 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.242637 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.242727 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.242946 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.243037 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.243252 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.243337 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.243555 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.243641 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.243874 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.243957 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.244182 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.244264 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.244483 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.244574 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.244810 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.244897 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.245133 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.245220 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.245449 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.245535 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.245754 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.245837 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.246063 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.246156 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.246374 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.246458 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.246675 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.246759 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.246986 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.247083 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.247290 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.247380 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.247596 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.247680 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.247897 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.247992 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.248198 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.248282 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.248503 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.248592 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.248929 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.249037 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.249239 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.249324 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.249557 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.249644 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.249869 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.249955 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.250179 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.250270 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.250477 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.250559 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.250785 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.250869 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.251090 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.251174 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.251391 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.251473 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.251696 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.251786 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.252003 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.252099 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.252304 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.252394 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.252610 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.252729 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.252937 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.253029 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.253246 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.253338 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.253547 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.253631 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.253848 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.253972 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.254155 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.254284 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.254461 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.254594 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.254761 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.254899 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.255084 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.255210 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.255390 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.255524 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.255694 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.255838 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.256001 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.256150 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.256302 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.256453 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.256614 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.256895 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.256994 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.257211 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.257349 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.257524 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.257660 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.257837 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.257971 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.258144 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.258274 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.258448 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.258571 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.258758 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.258882 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.259084 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.259195 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.259390 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.259508 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.259696 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.259811 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.259998 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.260222 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.260315 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.260530 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.260625 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.260861 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.260948 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.261172 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.261360 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.261475 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.261676 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.261762 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.261984 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.262074 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.262383 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.262465 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.262685 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.262770 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.262986 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.263069 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.263287 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.263369 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.263606 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.263691 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.263907 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.263992 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.264220 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.264309 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.264559 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.264646 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.264897 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.264983 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.265224 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.265324 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.265554 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.265661 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.265868 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.265971 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.266184 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.266270 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.266490 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.266586 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.266814 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.266899 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.267138 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.267221 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.267452 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.267549 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.267778 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.267861 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.268085 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.268190 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.268406 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.268503 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.268744 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.268832 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.269116 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.269211 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.269440 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.269526 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.269743 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.269828 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.270047 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.270137 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.270359 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.270445 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.270666 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.270748 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.270971 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.271054 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.271296 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.271381 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.271600 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.271683 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.271911 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.271995 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.272224 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.272308 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.272533 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.272616 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.272861 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.272960 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.273168 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.273259 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.273479 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.273567 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.273782 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.273873 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.274081 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.274175 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.274400 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.274485 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.274704 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.274797 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.275011 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.275097 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.275317 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.275400 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.275622 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.275716 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.275923 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.276015 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.276224 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.276308 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.276527 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.276611 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.276858 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.276944 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.277181 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.277271 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.277491 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.277576 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.277809 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.277893 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.278127 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.278212 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.278434 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.278519 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.278749 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.278833 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.279055 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.279138 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.279359 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.279452 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.279661 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.279789 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.279963 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.280559 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.280839 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.280939 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.281144 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.281236 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.281447 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.281539 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.281758 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.281850 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.282064 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.282156 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.282371 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.282464 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.282677 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.282770 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.282987 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.283071 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.283293 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.283385 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.283597 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.283682 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.283900 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.283983 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.284209 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.284294 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.284513 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.284597 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.284847 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.284951 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.285151 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.285237 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.285453 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.285536 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.285754 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.285836 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.286060 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.286144 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.286358 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.286441 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.286660 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.286753 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.286965 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.287052 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.287284 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.287372 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.287584 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.287677 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.287905 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.288001 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.288211 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.288304 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.288513 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.288606 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.288837 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.288925 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.289141 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.289227 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.289467 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.289559 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.289768 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.289896 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.290082 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.290173 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.290395 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.290488 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.290709 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.290802 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.291019 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.291104 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.291326 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.291417 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.291636 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.291719 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.291946 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.292029 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.292261 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.292460 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.292574 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.292815 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.292903 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.293133 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.293230 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.293448 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.293533 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.293761 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.293846 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.294075 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.294169 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.294389 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.294474 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.294699 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.294783 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.295012 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.295104 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.295322 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.295518 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.295633 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.296424 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.296527 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.296763 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.296852 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.297089 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.297183 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.297402 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.297496 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.297715 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.297800 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.298026 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.298110 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.298335 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.298422 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.298644 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.298729 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.298953 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.299039 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.299263 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.299356 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.299587 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.299679 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.299906 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.300064 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.300221 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.300313 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.300846 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.300937 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.301163 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.301264 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.301474 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.301558 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.301787 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.301871 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.302098 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.302190 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.302407 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.302492 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.302718 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.302803 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.303031 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.303117 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.303340 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.303465 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.303653 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.303738 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.303965 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.304052 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.304274 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.304370 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.304582 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.304675 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.304918 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.305007 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.305228 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.305312 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.305536 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.305746 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.305835 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.306066 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.306159 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.306406 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.306490 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.306737 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.306831 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.307064 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.307149 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.307378 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.307469 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.307690 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.307776 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.308003 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.308087 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.308308 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.308401 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.308617 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.308841 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.308940 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.309156 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.309240 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.309487 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.309573 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.309809 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.309893 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.310132 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.310217 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.310443 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.310527 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.310755 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.310839 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.311062 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.311146 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.311395 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.311616 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.311701 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.311960 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.312044 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.312277 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.312363 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.312574 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.312657 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.312941 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.313034 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.313248 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.313332 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.313550 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.313636 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.313867 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.313951 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.314173 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.314257 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.314482 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.314694 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.314799 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.315034 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.315127 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.315348 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.315433 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.315671 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.315758 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.315990 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.316075 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.316301 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.316383 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.316605 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.316687 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.316944 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.317032 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.317247 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.317331 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.317552 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.317636 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.317863 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.317949 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.318164 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.318247 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.318464 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.318546 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.318766 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.318852 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.319071 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.319200 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.319375 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.319474 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.319673 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.319756 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.319973 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.320056 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.320270 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.320368 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.320574 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.320659 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.320908 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.321000 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.321212 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.321305 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.321514 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.321605 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.321818 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.321909 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.322118 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.322214 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.322416 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.322508 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.322717 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.322802 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.323017 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.323103 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.323326 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.323412 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.323625 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.323709 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.323930 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.324016 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.324232 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.324316 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.324540 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.324623 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.324884 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.324971 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.325191 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.325284 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.325537 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.325621 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.325855 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.326069 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.326154 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.326412 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.326497 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.326734 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.326819 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.327552 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.327644 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.327872 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.327957 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.328182 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.328269 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.328488 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.328572 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.328819 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.328908 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.329147 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.329234 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.329460 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.329672 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.329758 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.330006 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.330099 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.330326 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.330427 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.330656 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.330741 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.330963 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.331048 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.331273 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.331364 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.331575 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.331665 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.331883 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.331977 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.332193 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.332277 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.332503 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.332588 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.332836 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.332924 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.333149 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.333347 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.333465 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.333807 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.333893 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.334128 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.334432 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.334516 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.334749 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.334873 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.335067 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.335192 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.335376 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.335509 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.335685 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.335822 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.335993 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.336133 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.336300 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.336445 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.336615 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.336776 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.336973 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.337098 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.337290 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.337411 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.337600 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.337752 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.338018 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.338111 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.338332 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.338416 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.338651 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.338745 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.338971 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.339058 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.339280 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.339413 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.339590 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.339675 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.339912 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.340197 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.340282 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.340519 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.340632 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.340859 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.340956 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.341191 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.341285 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.341516 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.341610 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.341811 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.341911 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.342164 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.342258 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.342472 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.342576 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.343324 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.343415 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.343669 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.343764 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.343984 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.344076 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.344304 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.344390 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.344615 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.344727 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.344946 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.345043 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.345254 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.345339 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.345567 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.345659 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.345873 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.345957 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.346179 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.346263 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.346485 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.346570 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.346794 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.346878 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.347103 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.347194 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.347407 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.347493 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.347711 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.347855 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.348015 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.348101 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.348317 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.348404 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.348622 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.348739 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.348948 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.349044 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.349250 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.349335 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.349552 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.349626 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.349915 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.350022 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.350149 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.350234 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.350446 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.350529 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.350746 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.350837 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.351049 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.351132 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.351348 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.351433 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.351648 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.351746 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.351950 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.352034 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.352251 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.352337 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.352552 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.352638 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.352900 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.352987 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.353208 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.353292 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.353510 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.353597 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.353812 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.353897 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.354113 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.354197 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.354414 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.354501 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.354717 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.354803 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.355045 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.355130 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.355356 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.355441 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.355670 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.355755 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.355982 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.356075 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.356288 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.356372 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.356607 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.356716 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.356944 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.357041 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.357258 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.357343 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.357574 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.357658 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.357886 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.357979 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.358211 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.358300 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.358531 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.358623 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.358842 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.358934 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.359152 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.359237 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.359461 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.359553 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.359770 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.359856 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.360077 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.360198 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.360387 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.360478 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.360725 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.360814 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.361031 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.361117 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.361337 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.361420 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.361648 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.361734 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.361964 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.362048 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.362275 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.362368 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.362588 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.362674 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.362898 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.362983 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.363209 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.363302 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.363522 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.363606 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.363831 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.363916 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.364139 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.364233 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.364447 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.364540 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.364775 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.364864 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.365092 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.365180 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.365402 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.365488 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.365711 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.365803 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.366024 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.366372 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.366548 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 19 10:55:38.366640 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.366916 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.367238 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.367552 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.367876 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.368201 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.368544 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.368881 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.369222 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.369556 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.369868 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.370279 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.370589 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 10:55:38.370831 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 19 10:55:38.429028 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO dbcounter [None req-5d58642d-c2da-4033-8784-9beec2ce965e None None] Registered counter for database nova_api Jul 19 10:55:38.432898 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO dbcounter [None req-6bd75e30-9b0d-47ba-ba66-7a6fff3bfa76 None None] Registered counter for database nova_api Jul 19 10:55:38.540249 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG oslo_db.sqlalchemy.engines [None req-5d58642d-c2da-4033-8784-9beec2ce965e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103328) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 19 10:55:38.544287 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG oslo_db.sqlalchemy.engines [None req-6bd75e30-9b0d-47ba-ba66-7a6fff3bfa76 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103329) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 19 10:55:38.546642 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG dbcounter [-] [103328] Writer thread running {{(pid=103328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 19 10:55:38.550661 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG dbcounter [-] [103329] Writer thread running {{(pid=103329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 19 10:55:38.602542 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.context [None req-5d58642d-c2da-4033-8784-9beec2ce965e None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),8c617d32-2606-4a09-a34c-e793c0a350a1(cell1) {{(pid=103328) load_cells /opt/stack/nova/nova/context.py:464}} Jul 19 10:55:38.606337 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.context [None req-6bd75e30-9b0d-47ba-ba66-7a6fff3bfa76 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),8c617d32-2606-4a09-a34c-e793c0a350a1(cell1) {{(pid=103329) load_cells /opt/stack/nova/nova/context.py:464}} Jul 19 10:55:38.606474 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG oslo_concurrency.lockutils [None req-5d58642d-c2da-4033-8784-9beec2ce965e None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103328) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 10:55:38.608041 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG oslo_concurrency.lockutils [None req-5d58642d-c2da-4033-8784-9beec2ce965e None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103328) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 10:55:38.608887 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG oslo_concurrency.lockutils [None req-5d58642d-c2da-4033-8784-9beec2ce965e None None] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103328) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 10:55:38.610244 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG oslo_concurrency.lockutils [None req-6bd75e30-9b0d-47ba-ba66-7a6fff3bfa76 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103329) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 10:55:38.610990 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG oslo_concurrency.lockutils [None req-5d58642d-c2da-4033-8784-9beec2ce965e None None] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103328) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 10:55:38.611815 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG oslo_concurrency.lockutils [None req-6bd75e30-9b0d-47ba-ba66-7a6fff3bfa76 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103329) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 10:55:38.612594 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG oslo_concurrency.lockutils [None req-6bd75e30-9b0d-47ba-ba66-7a6fff3bfa76 None None] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103329) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 10:55:38.612998 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO dbcounter [None req-5d58642d-c2da-4033-8784-9beec2ce965e None None] Registered counter for database nova_cell0 Jul 19 10:55:38.614708 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG oslo_concurrency.lockutils [None req-6bd75e30-9b0d-47ba-ba66-7a6fff3bfa76 None None] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103329) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 10:55:38.616968 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO dbcounter [None req-6bd75e30-9b0d-47ba-ba66-7a6fff3bfa76 None None] Registered counter for database nova_cell0 Jul 19 10:55:38.617107 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO dbcounter [None req-5d58642d-c2da-4033-8784-9beec2ce965e None None] Registered counter for database nova_cell1 Jul 19 10:55:38.620674 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO dbcounter [None req-6bd75e30-9b0d-47ba-ba66-7a6fff3bfa76 None None] Registered counter for database nova_cell1 Jul 19 10:55:38.625566 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG oslo_db.sqlalchemy.engines [None req-5d58642d-c2da-4033-8784-9beec2ce965e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103328) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 19 10:55:38.627282 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG oslo_db.sqlalchemy.engines [None req-5d58642d-c2da-4033-8784-9beec2ce965e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103328) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 19 10:55:38.628057 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG oslo_db.sqlalchemy.engines [None req-6bd75e30-9b0d-47ba-ba66-7a6fff3bfa76 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103329) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 19 10:55:38.629700 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG oslo_db.sqlalchemy.engines [None req-6bd75e30-9b0d-47ba-ba66-7a6fff3bfa76 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103329) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 19 10:55:38.632500 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG dbcounter [-] [103328] Writer thread running {{(pid=103328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 19 10:55:38.634624 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG dbcounter [-] [103329] Writer thread running {{(pid=103329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 19 10:55:38.634785 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG dbcounter [-] [103328] Writer thread running {{(pid=103328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 19 10:55:38.777064 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG dbcounter [-] [103329] Writer thread running {{(pid=103329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 19 10:55:38.784468 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO dbcounter [None req-5d58642d-c2da-4033-8784-9beec2ce965e None None] Registered counter for database nova_cell0 Jul 19 10:55:38.787205 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO dbcounter [None req-6bd75e30-9b0d-47ba-ba66-7a6fff3bfa76 None None] Registered counter for database nova_cell0 Jul 19 10:55:38.793478 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG oslo_db.sqlalchemy.engines [None req-5d58642d-c2da-4033-8784-9beec2ce965e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103328) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 19 10:55:38.795257 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG oslo_db.sqlalchemy.engines [None req-6bd75e30-9b0d-47ba-ba66-7a6fff3bfa76 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103329) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 19 10:55:38.797941 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG dbcounter [-] [103328] Writer thread running {{(pid=103328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 19 10:55:38.799629 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG dbcounter [-] [103329] Writer thread running {{(pid=103329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 19 10:55:39.134607 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: WARNING keystonemiddleware.auth_token [None req-5d58642d-c2da-4033-8784-9beec2ce965e None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 19 10:55:39.136162 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: WARNING keystonemiddleware.auth_token [None req-6bd75e30-9b0d-47ba-ba66-7a6fff3bfa76 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 19 10:55:39.142303 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: WARNING keystonemiddleware.auth_token [None req-5d58642d-c2da-4033-8784-9beec2ce965e None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 19 10:55:39.144002 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: WARNING keystonemiddleware.auth_token [None req-6bd75e30-9b0d-47ba-ba66-7a6fff3bfa76 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 19 10:55:39.482844 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: WARNING keystonemiddleware.auth_token [None req-5d58642d-c2da-4033-8784-9beec2ce965e None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 19 10:55:39.485670 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: WARNING keystonemiddleware.auth_token [None req-6bd75e30-9b0d-47ba-ba66-7a6fff3bfa76 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 19 10:55:39.489490 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: WARNING keystonemiddleware.auth_token [None req-5d58642d-c2da-4033-8784-9beec2ce965e None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 19 10:55:39.491967 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: WARNING keystonemiddleware.auth_token [None req-6bd75e30-9b0d-47ba-ba66-7a6fff3bfa76 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 19 10:55:39.494968 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x558002c8d9c0 pid: 103328 (default app) Jul 19 10:55:39.497025 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x558002c8d9c0 pid: 103329 (default app) Jul 19 10:55:39.497149 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-5d58642d-c2da-4033-8784-9beec2ce965e None None] Calling method '>' {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:55:39.498244 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-5d58642d-c2da-4033-8784-9beec2ce965e None None] 10.222.0.33 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001760 Jul 19 10:55:39.498662 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 1/1] 10.222.0.33 () {50 vars in 788 bytes} [Tue Jul 19 10:55:39 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 10:56:54.657427 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-6bd75e30-9b0d-47ba-ba66-7a6fff3bfa76 None None] Calling method '>' {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:56:54.658756 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-6bd75e30-9b0d-47ba-ba66-7a6fff3bfa76 None None] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002111 Jul 19 10:56:54.659494 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 1/2] 10.222.0.33 () {54 vars in 930 bytes} [Tue Jul 19 10:56:54 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 10:56:54.671265 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG dbcounter [-] [103328] Writing DB stats nova_api:SELECT=3 {{(pid=103328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 10:56:54.672386 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG dbcounter [-] [103328] Writing DB stats nova_cell1:SELECT=3 {{(pid=103328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 10:56:54.673464 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG dbcounter [-] [103328] Writing DB stats nova_cell0:SELECT=3 {{(pid=103328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 10:56:54.674231 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG dbcounter [-] [103328] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=103328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 10:56:55.526188 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-228d348a-1003-4454-b0f4-5b75c66f08bd admin admin] Calling method '>' {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:56:55.558728 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-228d348a-1003-4454-b0f4-5b75c66f08bd admin admin] 10.222.0.33 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.892624 Jul 19 10:56:55.568781 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 2/3] 10.222.0.33 () {60 vars in 1291 bytes} [Tue Jul 19 10:56:54 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 894 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 10:56:57.840342 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-6bd75e30-9b0d-47ba-ba66-7a6fff3bfa76 None None] Calling method '>' {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:56:57.843815 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-6bd75e30-9b0d-47ba-ba66-7a6fff3bfa76 None None] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.003975 Jul 19 10:56:57.844280 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 2/4] 10.222.0.33 () {54 vars in 930 bytes} [Tue Jul 19 10:56:57 2022] GET /compute/v2.1 => generated 388 bytes in 5 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 10:56:58.275445 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-d92ef3e0-a451-4c6e-9820-5905dff88399 admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "vcpus": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "name": "cirros256", "rxtx_factor": 1.0, "ram": 256, "id": "c1", "disk": 1}} {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 10:56:58.294088 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 19 10:56:58.294088 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: warnings.warn(msg) Jul 19 10:56:58.321678 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-d92ef3e0-a451-4c6e-9820-5905dff88399 admin admin] 10.222.0.33 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.471041 Jul 19 10:56:58.322420 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 3/5] 10.222.0.33 () {64 vars in 1282 bytes} [Tue Jul 19 10:56:57 2022] POST /compute/v2.1/flavors => generated 407 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 19 10:56:58.335824 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG dbcounter [-] [103329] Writing DB stats nova_api:SELECT=3 {{(pid=103329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 10:56:58.336970 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG dbcounter [-] [103329] Writing DB stats nova_cell0:SELECT=3 {{(pid=103329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 10:56:58.337688 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG dbcounter [-] [103329] Writing DB stats nova_cell1:SELECT=3 {{(pid=103329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 10:56:58.338594 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG dbcounter [-] [103329] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=103329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 10:56:58.548753 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-e332140c-9b02-4c21-aefb-abd12d27bb1b admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 10:56:58.551379 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 19 10:56:58.551379 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: warnings.warn(msg) Jul 19 10:56:58.620063 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-e332140c-9b02-4c21-aefb-abd12d27bb1b admin admin] 10.222.0.33 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.289616 Jul 19 10:56:58.620881 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 3/6] 10.222.0.33 () {64 vars in 1335 bytes} [Tue Jul 19 10:56:58 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 10:57:00.885956 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-d92ef3e0-a451-4c6e-9820-5905dff88399 admin admin] Calling method '>' {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:57:00.886971 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-d92ef3e0-a451-4c6e-9820-5905dff88399 admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001452 Jul 19 10:57:00.887366 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 4/7] 10.222.0.33 () {54 vars in 930 bytes} [Tue Jul 19 10:57:00 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 10:57:01.549627 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-ce586a8c-db6e-4964-993a-747eb3c11026 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "name": "ds512M", "ram": 512, "disk": 5, "id": "d1"}} {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 10:57:01.567267 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 19 10:57:01.567267 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: warnings.warn(msg) Jul 19 10:57:01.587598 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-ce586a8c-db6e-4964-993a-747eb3c11026 admin admin] 10.222.0.33 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.693743 Jul 19 10:57:01.595070 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 4/8] 10.222.0.33 () {64 vars in 1282 bytes} [Tue Jul 19 10:57:00 2022] POST /compute/v2.1/flavors => generated 404 bytes in 695 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 19 10:57:01.605423 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-4aeb7888-6c5c-45e8-aae4-f75536a344aa admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 10:57:01.607676 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 19 10:57:01.607676 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: warnings.warn(msg) Jul 19 10:57:01.646972 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-4aeb7888-6c5c-45e8-aae4-f75536a344aa admin admin] 10.222.0.33 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.045405 Jul 19 10:57:01.647554 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 5/9] 10.222.0.33 () {64 vars in 1335 bytes} [Tue Jul 19 10:57:01 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 10:57:03.906205 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-ce586a8c-db6e-4964-993a-747eb3c11026 admin admin] Calling method '>' {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:57:03.907222 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-ce586a8c-db6e-4964-993a-747eb3c11026 admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001461 Jul 19 10:57:03.907632 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 5/10] 10.222.0.33 () {54 vars in 930 bytes} [Tue Jul 19 10:57:03 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 10:57:04.322740 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-acafac09-4488-47aa-abc6-e360a0acb8e0 admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 1024, "swap": 0, "id": "d2", "disk": 10, "os-flavor-access:is_public": true, "name": "ds1G", "vcpus": 1}} {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 10:57:04.340493 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-acafac09-4488-47aa-abc6-e360a0acb8e0 admin admin] 10.222.0.33 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.425356 Jul 19 10:57:04.341016 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 6/11] 10.222.0.33 () {64 vars in 1282 bytes} [Tue Jul 19 10:57:03 2022] POST /compute/v2.1/flavors => generated 404 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 19 10:57:04.353039 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-af243206-bb01-48a4-86eb-9e73e4d26330 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 10:57:04.375555 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-af243206-bb01-48a4-86eb-9e73e4d26330 admin admin] 10.222.0.33 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.026257 Jul 19 10:57:04.376186 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 6/12] 10.222.0.33 () {64 vars in 1335 bytes} [Tue Jul 19 10:57:04 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 10:57:06.608365 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-acafac09-4488-47aa-abc6-e360a0acb8e0 admin admin] Calling method '>' {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:57:06.609347 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-acafac09-4488-47aa-abc6-e360a0acb8e0 admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001495 Jul 19 10:57:06.609751 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 7/13] 10.222.0.33 () {54 vars in 930 bytes} [Tue Jul 19 10:57:06 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 10:57:07.055210 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-bef42123-2ffe-423b-ba72-aeb3aa79127e admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "disk": 10, "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 2, "rxtx_factor": 1.0, "id": "d3", "ram": 2048, "name": "ds2G", "swap": 0}} {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 10:57:07.077377 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-bef42123-2ffe-423b-ba72-aeb3aa79127e admin admin] 10.222.0.33 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.461630 Jul 19 10:57:07.078193 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 7/14] 10.222.0.33 () {64 vars in 1282 bytes} [Tue Jul 19 10:57:06 2022] POST /compute/v2.1/flavors => generated 404 bytes in 463 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 19 10:57:07.091596 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-a00676d0-98c4-4de6-868c-c9ebaee33ddd admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 10:57:07.121204 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-a00676d0-98c4-4de6-868c-c9ebaee33ddd admin admin] 10.222.0.33 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.034281 Jul 19 10:57:07.121964 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 8/15] 10.222.0.33 () {64 vars in 1335 bytes} [Tue Jul 19 10:57:07 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 10:57:09.357974 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-bef42123-2ffe-423b-ba72-aeb3aa79127e admin admin] Calling method '>' {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:57:09.358977 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-bef42123-2ffe-423b-ba72-aeb3aa79127e admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001431 Jul 19 10:57:09.359633 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 8/16] 10.222.0.33 () {54 vars in 930 bytes} [Tue Jul 19 10:57:09 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 10:57:09.785650 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-7f34a6a8-7bd9-4f74-bf70-bae892399069 admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "d4", "swap": 0, "ram": 4096, "vcpus": 4, "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 20, "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "name": "ds4G"}} {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 10:57:09.803121 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-7f34a6a8-7bd9-4f74-bf70-bae892399069 admin admin] 10.222.0.33 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.436580 Jul 19 10:57:09.803762 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 9/17] 10.222.0.33 () {64 vars in 1282 bytes} [Tue Jul 19 10:57:09 2022] POST /compute/v2.1/flavors => generated 404 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 19 10:57:09.815911 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-0ca31832-1b89-4341-8b4c-6f6d0d7e3d8c admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 10:57:09.839110 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-0ca31832-1b89-4341-8b4c-6f6d0d7e3d8c admin admin] 10.222.0.33 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.026742 Jul 19 10:57:09.839920 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 9/18] 10.222.0.33 () {64 vars in 1335 bytes} [Tue Jul 19 10:57:09 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 10:57:12.253817 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-7f34a6a8-7bd9-4f74-bf70-bae892399069 admin admin] Calling method '>' {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:57:12.254823 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-7f34a6a8-7bd9-4f74-bf70-bae892399069 admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001441 Jul 19 10:57:12.255220 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 10/19] 10.222.0.33 () {54 vars in 930 bytes} [Tue Jul 19 10:57:12 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 10:57:12.698481 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-73e0427e-81c5-4c48-ac3b-6ecd5ccf6993 admin admin] Calling method '>' {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:57:12.724263 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-73e0427e-81c5-4c48-ac3b-6ecd5ccf6993 admin admin] 10.222.0.33 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.462870 Jul 19 10:57:12.724850 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 10/20] 10.222.0.33 () {60 vars in 1291 bytes} [Tue Jul 19 10:57:12 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 464 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 10:57:14.972737 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-7f34a6a8-7bd9-4f74-bf70-bae892399069 admin admin] Calling method '>' {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:57:14.973837 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-7f34a6a8-7bd9-4f74-bf70-bae892399069 admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001709 Jul 19 10:57:14.974351 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 11/21] 10.222.0.33 () {54 vars in 930 bytes} [Tue Jul 19 10:57:14 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 10:57:15.376290 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-50908b39-ed21-4653-bcd3-bac880f30e85 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "id": "1", "vcpus": 1, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "disk": 1, "swap": 0, "name": "m1.tiny"}} {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 10:57:15.401164 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-50908b39-ed21-4653-bcd3-bac880f30e85 admin admin] 10.222.0.33 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.420999 Jul 19 10:57:15.401729 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 11/22] 10.222.0.33 () {64 vars in 1282 bytes} [Tue Jul 19 10:57:14 2022] POST /compute/v2.1/flavors => generated 402 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 19 10:57:15.414697 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-9c6907ce-7de9-4610-a84b-c32f360bd186 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 10:57:15.439926 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-9c6907ce-7de9-4610-a84b-c32f360bd186 admin admin] 10.222.0.33 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.029266 Jul 19 10:57:15.440807 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 12/23] 10.222.0.33 () {64 vars in 1332 bytes} [Tue Jul 19 10:57:15 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 10:57:17.739257 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-50908b39-ed21-4653-bcd3-bac880f30e85 admin admin] Calling method '>' {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:57:17.740270 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-50908b39-ed21-4653-bcd3-bac880f30e85 admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001462 Jul 19 10:57:17.740675 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 12/24] 10.222.0.33 () {54 vars in 930 bytes} [Tue Jul 19 10:57:17 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 10:57:18.152842 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-130ff04f-bca3-424f-bbfa-b3e9b0493fe7 admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "disk": 20, "ram": 2048, "swap": 0, "vcpus": 1, "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "id": "2", "name": "m1.small"}} {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 10:57:18.170340 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-130ff04f-bca3-424f-bbfa-b3e9b0493fe7 admin admin] 10.222.0.33 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.423767 Jul 19 10:57:18.170888 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 13/25] 10.222.0.33 () {64 vars in 1282 bytes} [Tue Jul 19 10:57:17 2022] POST /compute/v2.1/flavors => generated 405 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 19 10:57:18.182669 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-bf0e509e-7cf7-45f0-a33b-9881530844cf admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 10:57:18.205657 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-bf0e509e-7cf7-45f0-a33b-9881530844cf admin admin] 10.222.0.33 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.026636 Jul 19 10:57:18.206220 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 13/26] 10.222.0.33 () {64 vars in 1332 bytes} [Tue Jul 19 10:57:18 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 10:57:20.549431 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-130ff04f-bca3-424f-bbfa-b3e9b0493fe7 admin admin] Calling method '>' {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:57:20.550478 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-130ff04f-bca3-424f-bbfa-b3e9b0493fe7 admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001515 Jul 19 10:57:20.550880 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 14/27] 10.222.0.33 () {54 vars in 930 bytes} [Tue Jul 19 10:57:20 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 10:57:20.975584 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-929d923a-050a-4d46-aba7-3ea4c2434e0d admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.medium", "vcpus": 2, "disk": 40, "rxtx_factor": 1.0, "swap": 0, "id": "3", "ram": 4096, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true}} {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 10:57:20.995556 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-929d923a-050a-4d46-aba7-3ea4c2434e0d admin admin] 10.222.0.33 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.438981 Jul 19 10:57:20.996091 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 14/28] 10.222.0.33 () {64 vars in 1282 bytes} [Tue Jul 19 10:57:20 2022] POST /compute/v2.1/flavors => generated 406 bytes in 440 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 19 10:57:21.008803 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-c3ec1f83-56e7-4195-a3c6-b43bc4c05bc7 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 10:57:21.031556 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-c3ec1f83-56e7-4195-a3c6-b43bc4c05bc7 admin admin] 10.222.0.33 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.026521 Jul 19 10:57:21.032126 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 15/29] 10.222.0.33 () {64 vars in 1332 bytes} [Tue Jul 19 10:57:21 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 10:57:23.354872 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-929d923a-050a-4d46-aba7-3ea4c2434e0d admin admin] Calling method '>' {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:57:23.356540 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-929d923a-050a-4d46-aba7-3ea4c2434e0d admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001546 Jul 19 10:57:23.357158 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 15/30] 10.222.0.33 () {54 vars in 930 bytes} [Tue Jul 19 10:57:23 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 10:57:23.820427 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-2221f3b4-faaf-47a0-989d-1d6f866f5c33 admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "4", "os-flavor-access:is_public": true, "name": "m1.large", "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 8192, "vcpus": 4, "swap": 0, "disk": 80}} {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 10:57:23.842193 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-2221f3b4-faaf-47a0-989d-1d6f866f5c33 admin admin] 10.222.0.33 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.478483 Jul 19 10:57:23.842869 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 16/31] 10.222.0.33 () {64 vars in 1282 bytes} [Tue Jul 19 10:57:23 2022] POST /compute/v2.1/flavors => generated 405 bytes in 480 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 19 10:57:23.854615 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-405c87c6-52a5-4e3b-adc9-2b89659775fc admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 10:57:23.880581 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-405c87c6-52a5-4e3b-adc9-2b89659775fc admin admin] 10.222.0.33 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.029548 Jul 19 10:57:23.881313 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 16/32] 10.222.0.33 () {64 vars in 1332 bytes} [Tue Jul 19 10:57:23 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 10:57:26.132396 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-2221f3b4-faaf-47a0-989d-1d6f866f5c33 admin admin] Calling method '>' {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:57:26.133639 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-2221f3b4-faaf-47a0-989d-1d6f866f5c33 admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001549 Jul 19 10:57:26.134212 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 17/33] 10.222.0.33 () {54 vars in 930 bytes} [Tue Jul 19 10:57:26 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 10:57:26.538677 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-2ba40f91-49e0-40fb-9978-3f0ab2664c24 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 16384, "id": "5", "swap": 0, "rxtx_factor": 1.0, "disk": 160, "os-flavor-access:is_public": true, "name": "m1.xlarge", "vcpus": 8, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 10:57:26.555760 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-2ba40f91-49e0-40fb-9978-3f0ab2664c24 admin admin] 10.222.0.33 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.415185 Jul 19 10:57:26.556299 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 17/34] 10.222.0.33 () {64 vars in 1282 bytes} [Tue Jul 19 10:57:26 2022] POST /compute/v2.1/flavors => generated 408 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 19 10:57:26.568780 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-193aa3e5-c466-47b0-8e46-fe38ff15581d admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 10:57:26.572255 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG dbcounter [-] [103328] Writing DB stats nova_api:SELECT=42,nova_api:INSERT=9,nova_api:SAVEPOINT=4,nova_api:RELEASE=4 {{(pid=103328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 10:57:26.594712 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-193aa3e5-c466-47b0-8e46-fe38ff15581d admin admin] 10.222.0.33 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.029734 Jul 19 10:57:26.596866 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 18/35] 10.222.0.33 () {64 vars in 1332 bytes} [Tue Jul 19 10:57:26 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 10:57:53.100523 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-2ba40f91-49e0-40fb-9978-3f0ab2664c24 admin admin] Calling method '>' {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:57:53.101556 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-2ba40f91-49e0-40fb-9978-3f0ab2664c24 admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001459 Jul 19 10:57:53.101951 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 18/36] 10.222.0.33 () {54 vars in 930 bytes} [Tue Jul 19 10:57:53 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 10:57:53.112860 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG dbcounter [-] [103328] Writing DB stats nova_api:SELECT=5,nova_api:SAVEPOINT=1,nova_api:INSERT=1,nova_api:RELEASE=1 {{(pid=103328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 10:57:53.484745 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-e619443c-02e5-467c-bdbd-664c60e7b934 admin admin] Calling method '>' {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:57:53.501653 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-e619443c-02e5-467c-bdbd-664c60e7b934 admin admin] 10.222.0.33 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.393332 Jul 19 10:57:53.502182 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 19/37] 10.222.0.33 () {60 vars in 1291 bytes} [Tue Jul 19 10:57:53 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 10:57:54.551519 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-2ba40f91-49e0-40fb-9978-3f0ab2664c24 admin admin] Calling method '>' {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:57:54.552524 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-2ba40f91-49e0-40fb-9978-3f0ab2664c24 admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001413 Jul 19 10:57:54.552945 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 19/38] 10.222.0.33 () {56 vars in 1074 bytes} [Tue Jul 19 10:57:54 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 10:57:55.073124 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-b9052b99-6042-4f32-9042-35ffb6567c84 admin admin] Calling method '>' {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:57:55.078629 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.wsgi [None req-b9052b99-6042-4f32-9042-35ffb6567c84 admin admin] HTTP exception thrown: Flavor 42 could not be found. Jul 19 10:57:55.082210 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-b9052b99-6042-4f32-9042-35ffb6567c84 admin admin] Request method failure captured: Jul 19 10:57:55.082210 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Jul 19 10:57:55.082210 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: Accept: application/json Jul 19 10:57:55.082210 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: Accept-Encoding: gzip, deflate Jul 19 10:57:55.082210 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: Connection: keep-alive Jul 19 10:57:55.082210 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: Content-Length: 0 Jul 19 10:57:55.082210 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: Host: 10.222.0.33 Jul 19 10:57:55.082210 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: Openstack-Api-Version: compute 2.90 Jul 19 10:57:55.082210 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: Openstack-System-Scope: None Jul 19 10:57:55.082210 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: User-Agent: python-novaclient Jul 19 10:57:55.082210 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Auth-Token: gAAAAABi1o4ySZXaJUnMnqHhom7uxXpCW60lPQYHbRU701deB3YyHfLcmk1R_7IgVyaVMTng1bi0btWpnw5kxOZIFdNgLjWNXd9kWQa7Mnydj266yG00lqDBnp-paDDdep2heYwBOpmI270-2LIxQ9h8vUoR_DznDXWlKdQkMjvc12e3EwI3Ve8 Jul 19 10:57:55.082210 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Domain-Id: None Jul 19 10:57:55.082210 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Domain-Name: None Jul 19 10:57:55.082210 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Identity-Status: Confirmed Jul 19 10:57:55.082210 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Is-Admin-Project: True Jul 19 10:57:55.082210 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Openstack-Nova-Api-Version: 2.90 Jul 19 10:57:55.082210 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Project-Domain-Id: default Jul 19 10:57:55.082210 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Project-Domain-Name: Default Jul 19 10:57:55.082210 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Project-Id: a1dc1eb7c79345caaeb191fa0f9c008f Jul 19 10:57:55.082210 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Project-Name: admin Jul 19 10:57:55.082210 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Role: member,reader,admin Jul 19 10:57:55.082210 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Roles: member,reader,admin Jul 19 10:57:55.083851 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a1dc1eb7c79345caaeb191fa0f9c008f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/a1dc1eb7c79345caaeb191fa0f9c008f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a1dc1eb7c79345caaeb191fa0f9c008f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_a1dc1eb7c79345caaeb191fa0f9c008f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 10:57:55.083851 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Tenant: admin Jul 19 10:57:55.083851 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Tenant-Id: a1dc1eb7c79345caaeb191fa0f9c008f Jul 19 10:57:55.083851 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Tenant-Name: admin Jul 19 10:57:55.083851 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-User: admin Jul 19 10:57:55.083851 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-User-Domain-Id: default Jul 19 10:57:55.083851 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-User-Domain-Name: Default Jul 19 10:57:55.083851 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-User-Id: 1eaf5dcf504f41ff99cb8d55073af0f9 Jul 19 10:57:55.083851 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-User-Name: admin Jul 19 10:57:55.083851 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: method: > Jul 19 10:57:55.083851 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: exception: Flavor 42 could not be found. Jul 19 10:57:55.083851 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: action_args: {'id': '42'} Jul 19 10:57:55.083851 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 10:57:55.083851 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 10:57:55.084880 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 19 10:57:55.084880 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 19 10:57:55.084880 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 19 10:57:55.084880 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 19 10:57:55.084880 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 19 10:57:55.084880 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 19 10:57:55.084880 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 19 10:57:55.084880 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 19 10:57:55.084880 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 19 10:57:55.084880 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 10:57:55.084880 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 19 10:57:55.084880 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 19 10:57:55.084880 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Jul 19 10:57:55.084880 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi Jul 19 10:57:55.084880 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 10:57:55.084880 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi Jul 19 10:57:55.084880 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 10:57:55.084880 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 10:57:55.084880 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 10:57:55.086024 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 10:57:55.086024 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 10:57:55.086024 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 10:57:55.086024 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 10:57:55.086024 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 19 10:57:55.086024 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 19 10:57:55.086024 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Jul 19 10:57:55.086024 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi Jul 19 10:57:55.086024 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 10:57:55.086024 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi Jul 19 10:57:55.086024 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 10:57:55.086024 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 10:57:55.086024 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 10:57:55.086024 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 10:57:55.086024 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 10:57:55.086024 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Jul 19 10:57:55.086024 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi Jul 19 10:57:55.086024 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-b9052b99-6042-4f32-9042-35ffb6567c84 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=103328) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 10:57:55.087589 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-b9052b99-6042-4f32-9042-35ffb6567c84 admin admin] 10.222.0.33 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.392049 Jul 19 10:57:55.087589 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 20/39] 10.222.0.33 () {60 vars in 1190 bytes} [Tue Jul 19 10:57:54 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 393 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jul 19 10:57:55.093254 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-1dcaefaf-6987-41f4-addc-b36e16c0bf08 admin admin] Calling method '>' {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:57:55.096170 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG dbcounter [-] [103329] Writing DB stats nova_api:SELECT=47,nova_api:SAVEPOINT=5,nova_api:INSERT=10,nova_api:RELEASE=5 {{(pid=103329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 10:57:55.115743 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-1dcaefaf-6987-41f4-addc-b36e16c0bf08 admin admin] 10.222.0.33 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.026021 Jul 19 10:57:55.116277 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 20/40] 10.222.0.33 () {60 vars in 1210 bytes} [Tue Jul 19 10:57:55 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 10:57:55.126709 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-60c1ccb1-9ec6-4cb2-b4c7-afcdf8e4f2f5 admin admin] Calling method '>' {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:57:55.147329 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-60c1ccb1-9ec6-4cb2-b4c7-afcdf8e4f2f5 admin admin] 10.222.0.33 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.024190 Jul 19 10:57:55.147860 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 21/41] 10.222.0.33 () {60 vars in 1210 bytes} [Tue Jul 19 10:57:55 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 25 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 10:57:55.156841 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-08034b0a-39c6-46e2-9715-97a477da43c2 admin admin] Calling method '>' {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:57:55.162701 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.wsgi [None req-08034b0a-39c6-46e2-9715-97a477da43c2 admin admin] HTTP exception thrown: Flavor 42 could not be found. Jul 19 10:57:55.166010 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-08034b0a-39c6-46e2-9715-97a477da43c2 admin admin] Request method failure captured: Jul 19 10:57:55.166010 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Jul 19 10:57:55.166010 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: Accept: application/json Jul 19 10:57:55.166010 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: Accept-Encoding: gzip, deflate Jul 19 10:57:55.166010 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: Connection: keep-alive Jul 19 10:57:55.166010 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: Content-Length: 0 Jul 19 10:57:55.166010 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: Host: 10.222.0.33 Jul 19 10:57:55.166010 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: Openstack-Api-Version: compute 2.90 Jul 19 10:57:55.166010 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: Openstack-System-Scope: None Jul 19 10:57:55.166010 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: User-Agent: python-novaclient Jul 19 10:57:55.166010 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Auth-Token: gAAAAABi1o4ySZXaJUnMnqHhom7uxXpCW60lPQYHbRU701deB3YyHfLcmk1R_7IgVyaVMTng1bi0btWpnw5kxOZIFdNgLjWNXd9kWQa7Mnydj266yG00lqDBnp-paDDdep2heYwBOpmI270-2LIxQ9h8vUoR_DznDXWlKdQkMjvc12e3EwI3Ve8 Jul 19 10:57:55.166010 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Domain-Id: None Jul 19 10:57:55.166010 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Domain-Name: None Jul 19 10:57:55.166010 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Identity-Status: Confirmed Jul 19 10:57:55.166010 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Is-Admin-Project: True Jul 19 10:57:55.166010 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Openstack-Nova-Api-Version: 2.90 Jul 19 10:57:55.166010 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Project-Domain-Id: default Jul 19 10:57:55.166010 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Project-Domain-Name: Default Jul 19 10:57:55.166010 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Project-Id: a1dc1eb7c79345caaeb191fa0f9c008f Jul 19 10:57:55.166010 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Project-Name: admin Jul 19 10:57:55.166010 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Role: member,reader,admin Jul 19 10:57:55.166010 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Roles: member,reader,admin Jul 19 10:57:55.167344 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a1dc1eb7c79345caaeb191fa0f9c008f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/a1dc1eb7c79345caaeb191fa0f9c008f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a1dc1eb7c79345caaeb191fa0f9c008f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_a1dc1eb7c79345caaeb191fa0f9c008f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 10:57:55.167344 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Tenant: admin Jul 19 10:57:55.167344 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Tenant-Id: a1dc1eb7c79345caaeb191fa0f9c008f Jul 19 10:57:55.167344 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Tenant-Name: admin Jul 19 10:57:55.167344 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-User: admin Jul 19 10:57:55.167344 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-User-Domain-Id: default Jul 19 10:57:55.167344 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-User-Domain-Name: Default Jul 19 10:57:55.167344 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-User-Id: 1eaf5dcf504f41ff99cb8d55073af0f9 Jul 19 10:57:55.167344 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-User-Name: admin Jul 19 10:57:55.167344 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: method: > Jul 19 10:57:55.167344 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: exception: Flavor 42 could not be found. Jul 19 10:57:55.167344 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: action_args: {'id': '42'} Jul 19 10:57:55.167344 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 10:57:55.167344 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 10:57:55.172437 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 19 10:57:55.172437 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 19 10:57:55.172437 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 19 10:57:55.172437 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 19 10:57:55.172437 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 19 10:57:55.172437 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 19 10:57:55.172437 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 19 10:57:55.172437 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 19 10:57:55.172437 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 19 10:57:55.172437 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 10:57:55.172437 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 19 10:57:55.172437 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 19 10:57:55.172437 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Jul 19 10:57:55.172437 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi Jul 19 10:57:55.172437 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 10:57:55.172437 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi Jul 19 10:57:55.172437 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 10:57:55.172437 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 10:57:55.172437 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 10:57:55.173709 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 10:57:55.173709 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 10:57:55.173709 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 10:57:55.173709 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 10:57:55.173709 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 19 10:57:55.173709 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 19 10:57:55.173709 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Jul 19 10:57:55.173709 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi Jul 19 10:57:55.173709 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 10:57:55.173709 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi Jul 19 10:57:55.173709 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 10:57:55.173709 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 10:57:55.173709 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 10:57:55.173709 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 10:57:55.173709 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 10:57:55.173709 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Jul 19 10:57:55.173709 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi Jul 19 10:57:55.173709 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-08034b0a-39c6-46e2-9715-97a477da43c2 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=103329) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 10:57:55.174887 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-08034b0a-39c6-46e2-9715-97a477da43c2 admin admin] 10.222.0.33 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.013952 Jul 19 10:57:55.174887 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 21/42] 10.222.0.33 () {60 vars in 1190 bytes} [Tue Jul 19 10:57:55 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 15 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) Jul 19 10:57:56.134262 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-60c1ccb1-9ec6-4cb2-b4c7-afcdf8e4f2f5 admin admin] Calling method '>' {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:57:56.135274 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-60c1ccb1-9ec6-4cb2-b4c7-afcdf8e4f2f5 admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001473 Jul 19 10:57:56.135698 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 22/43] 10.222.0.33 () {56 vars in 1074 bytes} [Tue Jul 19 10:57:56 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 10:57:56.671802 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-c71fb9c1-3816-44d7-94c1-a8ee992565fb admin admin] Calling method '>' {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:57:56.677097 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.wsgi [None req-c71fb9c1-3816-44d7-94c1-a8ee992565fb admin admin] HTTP exception thrown: Flavor 84 could not be found. Jul 19 10:57:56.678610 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-c71fb9c1-3816-44d7-94c1-a8ee992565fb admin admin] Request method failure captured: Jul 19 10:57:56.678610 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Jul 19 10:57:56.678610 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: Accept: application/json Jul 19 10:57:56.678610 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: Accept-Encoding: gzip, deflate Jul 19 10:57:56.678610 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: Connection: keep-alive Jul 19 10:57:56.678610 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: Content-Length: 0 Jul 19 10:57:56.678610 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: Host: 10.222.0.33 Jul 19 10:57:56.678610 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: Openstack-Api-Version: compute 2.90 Jul 19 10:57:56.678610 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: Openstack-System-Scope: None Jul 19 10:57:56.678610 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: User-Agent: python-novaclient Jul 19 10:57:56.678610 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Auth-Token: gAAAAABi1o409L9DKhuzDRp6WHDEVoM8q4cm53qBqmNQ5ZJ_LqQ1ETiZhBpBx6duNyTbsFZv180MmfeVpdVzdev-_ALRgi5GuOvo-oyEsxYc5aLUY3f3PH1Xrx461_iklRyGmJEfsuEOpH5RbT8Asw7wOh6HDw0J4eqb12FN9HDFQ70sCZ3RERE Jul 19 10:57:56.678610 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Domain-Id: None Jul 19 10:57:56.678610 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Domain-Name: None Jul 19 10:57:56.678610 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Identity-Status: Confirmed Jul 19 10:57:56.678610 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Is-Admin-Project: True Jul 19 10:57:56.678610 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Openstack-Nova-Api-Version: 2.90 Jul 19 10:57:56.678610 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Project-Domain-Id: default Jul 19 10:57:56.678610 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Project-Domain-Name: Default Jul 19 10:57:56.678610 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Project-Id: a1dc1eb7c79345caaeb191fa0f9c008f Jul 19 10:57:56.678610 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Project-Name: admin Jul 19 10:57:56.678610 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Role: member,reader,admin Jul 19 10:57:56.678610 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Roles: member,reader,admin Jul 19 10:57:56.680156 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a1dc1eb7c79345caaeb191fa0f9c008f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/a1dc1eb7c79345caaeb191fa0f9c008f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a1dc1eb7c79345caaeb191fa0f9c008f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_a1dc1eb7c79345caaeb191fa0f9c008f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 10:57:56.680156 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Tenant: admin Jul 19 10:57:56.680156 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Tenant-Id: a1dc1eb7c79345caaeb191fa0f9c008f Jul 19 10:57:56.680156 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Tenant-Name: admin Jul 19 10:57:56.680156 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-User: admin Jul 19 10:57:56.680156 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-User-Domain-Id: default Jul 19 10:57:56.680156 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-User-Domain-Name: Default Jul 19 10:57:56.680156 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-User-Id: 1eaf5dcf504f41ff99cb8d55073af0f9 Jul 19 10:57:56.680156 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-User-Name: admin Jul 19 10:57:56.680156 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: method: > Jul 19 10:57:56.680156 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: exception: Flavor 84 could not be found. Jul 19 10:57:56.680156 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: action_args: {'id': '84'} Jul 19 10:57:56.680156 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 10:57:56.680156 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 10:57:56.681119 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 19 10:57:56.681119 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 19 10:57:56.681119 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 19 10:57:56.681119 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 19 10:57:56.681119 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 19 10:57:56.681119 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 19 10:57:56.681119 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 19 10:57:56.681119 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 19 10:57:56.681119 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 19 10:57:56.681119 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 10:57:56.681119 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 19 10:57:56.681119 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 19 10:57:56.681119 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Jul 19 10:57:56.681119 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi Jul 19 10:57:56.681119 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 10:57:56.681119 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi Jul 19 10:57:56.681119 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 10:57:56.681119 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 10:57:56.681119 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 10:57:56.682440 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 10:57:56.682440 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 10:57:56.682440 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 10:57:56.682440 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 10:57:56.682440 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 19 10:57:56.682440 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 19 10:57:56.682440 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Jul 19 10:57:56.682440 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi Jul 19 10:57:56.682440 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 10:57:56.682440 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi Jul 19 10:57:56.682440 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 10:57:56.682440 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 10:57:56.682440 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 10:57:56.682440 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 10:57:56.682440 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 10:57:56.682440 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Jul 19 10:57:56.682440 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi Jul 19 10:57:56.682440 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-c71fb9c1-3816-44d7-94c1-a8ee992565fb admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=103329) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 10:57:56.684288 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-c71fb9c1-3816-44d7-94c1-a8ee992565fb admin admin] 10.222.0.33 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.396299 Jul 19 10:57:56.684288 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 22/44] 10.222.0.33 () {60 vars in 1190 bytes} [Tue Jul 19 10:57:56 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 397 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jul 19 10:57:56.690143 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-c98b3bc1-296e-4b7b-91a1-69294b087fca admin admin] Calling method '>' {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:57:56.703419 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-c98b3bc1-296e-4b7b-91a1-69294b087fca admin admin] 10.222.0.33 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.016835 Jul 19 10:57:56.703931 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 23/45] 10.222.0.33 () {60 vars in 1210 bytes} [Tue Jul 19 10:57:56 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 18 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 10:57:56.712917 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-17058614-dc75-4bfd-9af1-2a9704fc7215 admin admin] Calling method '>' {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:57:56.726615 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-17058614-dc75-4bfd-9af1-2a9704fc7215 admin admin] 10.222.0.33 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.017172 Jul 19 10:57:56.727126 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 23/46] 10.222.0.33 () {60 vars in 1210 bytes} [Tue Jul 19 10:57:56 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 18 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 10:57:56.736450 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-5261ac65-449d-427a-9739-fa2480d7436d admin admin] Calling method '>' {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:57:56.741668 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.wsgi [None req-5261ac65-449d-427a-9739-fa2480d7436d admin admin] HTTP exception thrown: Flavor 84 could not be found. Jul 19 10:57:56.743422 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-5261ac65-449d-427a-9739-fa2480d7436d admin admin] Request method failure captured: Jul 19 10:57:56.743422 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Jul 19 10:57:56.743422 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: Accept: application/json Jul 19 10:57:56.743422 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: Accept-Encoding: gzip, deflate Jul 19 10:57:56.743422 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: Connection: keep-alive Jul 19 10:57:56.743422 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: Content-Length: 0 Jul 19 10:57:56.743422 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: Host: 10.222.0.33 Jul 19 10:57:56.743422 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: Openstack-Api-Version: compute 2.90 Jul 19 10:57:56.743422 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: Openstack-System-Scope: None Jul 19 10:57:56.743422 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: User-Agent: python-novaclient Jul 19 10:57:56.743422 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Auth-Token: gAAAAABi1o409L9DKhuzDRp6WHDEVoM8q4cm53qBqmNQ5ZJ_LqQ1ETiZhBpBx6duNyTbsFZv180MmfeVpdVzdev-_ALRgi5GuOvo-oyEsxYc5aLUY3f3PH1Xrx461_iklRyGmJEfsuEOpH5RbT8Asw7wOh6HDw0J4eqb12FN9HDFQ70sCZ3RERE Jul 19 10:57:56.743422 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Domain-Id: None Jul 19 10:57:56.743422 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Domain-Name: None Jul 19 10:57:56.743422 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Identity-Status: Confirmed Jul 19 10:57:56.743422 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Is-Admin-Project: True Jul 19 10:57:56.743422 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Openstack-Nova-Api-Version: 2.90 Jul 19 10:57:56.743422 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Project-Domain-Id: default Jul 19 10:57:56.743422 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Project-Domain-Name: Default Jul 19 10:57:56.743422 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Project-Id: a1dc1eb7c79345caaeb191fa0f9c008f Jul 19 10:57:56.743422 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Project-Name: admin Jul 19 10:57:56.743422 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Role: member,reader,admin Jul 19 10:57:56.743422 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Roles: member,reader,admin Jul 19 10:57:56.745059 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a1dc1eb7c79345caaeb191fa0f9c008f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/a1dc1eb7c79345caaeb191fa0f9c008f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a1dc1eb7c79345caaeb191fa0f9c008f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_a1dc1eb7c79345caaeb191fa0f9c008f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 10:57:56.745059 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Tenant: admin Jul 19 10:57:56.745059 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Tenant-Id: a1dc1eb7c79345caaeb191fa0f9c008f Jul 19 10:57:56.745059 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Tenant-Name: admin Jul 19 10:57:56.745059 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-User: admin Jul 19 10:57:56.745059 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-User-Domain-Id: default Jul 19 10:57:56.745059 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-User-Domain-Name: Default Jul 19 10:57:56.745059 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-User-Id: 1eaf5dcf504f41ff99cb8d55073af0f9 Jul 19 10:57:56.745059 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-User-Name: admin Jul 19 10:57:56.745059 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: method: > Jul 19 10:57:56.745059 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: exception: Flavor 84 could not be found. Jul 19 10:57:56.745059 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: action_args: {'id': '84'} Jul 19 10:57:56.745059 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 10:57:56.745059 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 10:57:56.746204 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 19 10:57:56.746204 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 19 10:57:56.746204 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 19 10:57:56.746204 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 19 10:57:56.746204 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 19 10:57:56.746204 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 19 10:57:56.746204 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 19 10:57:56.746204 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 19 10:57:56.746204 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 19 10:57:56.746204 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 10:57:56.746204 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 19 10:57:56.746204 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 19 10:57:56.746204 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Jul 19 10:57:56.746204 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi Jul 19 10:57:56.746204 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 10:57:56.746204 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi Jul 19 10:57:56.746204 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 10:57:56.746204 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 10:57:56.746204 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 10:57:56.747603 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 10:57:56.747603 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 10:57:56.747603 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 10:57:56.747603 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 10:57:56.747603 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 19 10:57:56.747603 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 19 10:57:56.747603 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Jul 19 10:57:56.747603 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi Jul 19 10:57:56.747603 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 10:57:56.747603 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi Jul 19 10:57:56.747603 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 10:57:56.747603 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 10:57:56.747603 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 10:57:56.747603 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 10:57:56.747603 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 10:57:56.747603 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Jul 19 10:57:56.747603 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi Jul 19 10:57:56.747603 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-5261ac65-449d-427a-9739-fa2480d7436d admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=103328) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 10:57:56.748938 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-5261ac65-449d-427a-9739-fa2480d7436d admin admin] 10.222.0.33 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.011684 Jul 19 10:57:56.748938 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 24/47] 10.222.0.33 () {60 vars in 1190 bytes} [Tue Jul 19 10:57:56 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) Jul 19 10:57:57.707110 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-17058614-dc75-4bfd-9af1-2a9704fc7215 admin admin] Calling method '>' {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:57:57.708115 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-17058614-dc75-4bfd-9af1-2a9704fc7215 admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001462 Jul 19 10:57:57.708513 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 24/48] 10.222.0.33 () {56 vars in 1074 bytes} [Tue Jul 19 10:57:57 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 10:57:58.240639 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-4ec26a5e-4c66-4bcc-b061-331ce0924be0 admin admin] Calling method '>' {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:57:58.247157 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.wsgi [None req-4ec26a5e-4c66-4bcc-b061-331ce0924be0 admin admin] HTTP exception thrown: Flavor 451 could not be found. Jul 19 10:57:58.248640 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-4ec26a5e-4c66-4bcc-b061-331ce0924be0 admin admin] Request method failure captured: Jul 19 10:57:58.248640 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Jul 19 10:57:58.248640 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: Accept: application/json Jul 19 10:57:58.248640 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: Accept-Encoding: gzip, deflate Jul 19 10:57:58.248640 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: Connection: keep-alive Jul 19 10:57:58.248640 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: Content-Length: 0 Jul 19 10:57:58.248640 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: Host: 10.222.0.33 Jul 19 10:57:58.248640 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: Openstack-Api-Version: compute 2.90 Jul 19 10:57:58.248640 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: Openstack-System-Scope: None Jul 19 10:57:58.248640 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: User-Agent: python-novaclient Jul 19 10:57:58.248640 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Auth-Token: gAAAAABi1o41d9kqTmnUaZJofBrHGjjzZ_PzEeCC0sW4KMI2YKwJh7uLW_LJt-yrjnrPlVk-kBrMpBCfyRUruBFjrEL3hci5sQ7ErlKp0km4OT9dVidhy7yIhn5rjspxfljYDSwp20GwsQreznrkqwhe1HJNqS8ig9z-oWLZymBsxnBt4AJ-hHo Jul 19 10:57:58.248640 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Domain-Id: None Jul 19 10:57:58.248640 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Domain-Name: None Jul 19 10:57:58.248640 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Identity-Status: Confirmed Jul 19 10:57:58.248640 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Is-Admin-Project: True Jul 19 10:57:58.248640 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Openstack-Nova-Api-Version: 2.90 Jul 19 10:57:58.248640 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Project-Domain-Id: default Jul 19 10:57:58.248640 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Project-Domain-Name: Default Jul 19 10:57:58.248640 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Project-Id: a1dc1eb7c79345caaeb191fa0f9c008f Jul 19 10:57:58.248640 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Project-Name: admin Jul 19 10:57:58.248640 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Role: member,reader,admin Jul 19 10:57:58.248640 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Roles: member,reader,admin Jul 19 10:57:58.250224 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a1dc1eb7c79345caaeb191fa0f9c008f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/a1dc1eb7c79345caaeb191fa0f9c008f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a1dc1eb7c79345caaeb191fa0f9c008f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_a1dc1eb7c79345caaeb191fa0f9c008f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 10:57:58.250224 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Tenant: admin Jul 19 10:57:58.250224 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Tenant-Id: a1dc1eb7c79345caaeb191fa0f9c008f Jul 19 10:57:58.250224 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-Tenant-Name: admin Jul 19 10:57:58.250224 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-User: admin Jul 19 10:57:58.250224 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-User-Domain-Id: default Jul 19 10:57:58.250224 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-User-Domain-Name: Default Jul 19 10:57:58.250224 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-User-Id: 1eaf5dcf504f41ff99cb8d55073af0f9 Jul 19 10:57:58.250224 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: X-User-Name: admin Jul 19 10:57:58.250224 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: method: > Jul 19 10:57:58.250224 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: exception: Flavor 451 could not be found. Jul 19 10:57:58.250224 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: action_args: {'id': '451'} Jul 19 10:57:58.250224 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 10:57:58.250224 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 10:57:58.251106 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 19 10:57:58.251106 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 19 10:57:58.251106 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 19 10:57:58.251106 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 19 10:57:58.251106 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 19 10:57:58.251106 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 19 10:57:58.251106 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 19 10:57:58.251106 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 19 10:57:58.251106 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 19 10:57:58.251106 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 10:57:58.251106 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 19 10:57:58.251106 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 19 10:57:58.251106 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Jul 19 10:57:58.251106 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi Jul 19 10:57:58.251106 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 10:57:58.251106 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi Jul 19 10:57:58.251106 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 10:57:58.251106 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 10:57:58.251106 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 10:57:58.252133 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 10:57:58.252133 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 10:57:58.252133 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 10:57:58.252133 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 10:57:58.252133 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 19 10:57:58.252133 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 19 10:57:58.252133 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Jul 19 10:57:58.252133 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi Jul 19 10:57:58.252133 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 10:57:58.252133 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi Jul 19 10:57:58.252133 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 10:57:58.252133 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 10:57:58.252133 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 10:57:58.252133 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 10:57:58.252133 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 10:57:58.252133 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Jul 19 10:57:58.252133 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: ERROR nova.api.openstack.wsgi Jul 19 10:57:58.252133 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-4ec26a5e-4c66-4bcc-b061-331ce0924be0 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=103328) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 10:57:58.253267 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-4ec26a5e-4c66-4bcc-b061-331ce0924be0 admin admin] 10.222.0.33 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.404562 Jul 19 10:57:58.253267 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 25/49] 10.222.0.33 () {60 vars in 1193 bytes} [Tue Jul 19 10:57:57 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 406 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jul 19 10:57:58.259502 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-f5c4cc40-1795-45ca-a509-15ac652bb3f0 admin admin] Calling method '>' {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:57:58.273471 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-f5c4cc40-1795-45ca-a509-15ac652bb3f0 admin admin] 10.222.0.33 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.017476 Jul 19 10:57:58.274150 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 25/50] 10.222.0.33 () {60 vars in 1210 bytes} [Tue Jul 19 10:57:58 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 10:57:58.283409 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-27d4d50c-2175-439d-833c-84be4bdb2322 admin admin] Calling method '>' {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:57:58.297930 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-27d4d50c-2175-439d-833c-84be4bdb2322 admin admin] 10.222.0.33 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.017973 Jul 19 10:57:58.298445 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 26/51] 10.222.0.33 () {60 vars in 1210 bytes} [Tue Jul 19 10:57:58 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 10:57:58.307505 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-380cd15a-8a5c-4d82-9e5d-4713b02f119b admin admin] Calling method '>' {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:57:58.313370 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.wsgi [None req-380cd15a-8a5c-4d82-9e5d-4713b02f119b admin admin] HTTP exception thrown: Flavor 451 could not be found. Jul 19 10:57:58.314860 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-380cd15a-8a5c-4d82-9e5d-4713b02f119b admin admin] Request method failure captured: Jul 19 10:57:58.314860 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Jul 19 10:57:58.314860 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: Accept: application/json Jul 19 10:57:58.314860 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: Accept-Encoding: gzip, deflate Jul 19 10:57:58.314860 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: Connection: keep-alive Jul 19 10:57:58.314860 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: Content-Length: 0 Jul 19 10:57:58.314860 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: Host: 10.222.0.33 Jul 19 10:57:58.314860 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: Openstack-Api-Version: compute 2.90 Jul 19 10:57:58.314860 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: Openstack-System-Scope: None Jul 19 10:57:58.314860 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: User-Agent: python-novaclient Jul 19 10:57:58.314860 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Auth-Token: gAAAAABi1o41d9kqTmnUaZJofBrHGjjzZ_PzEeCC0sW4KMI2YKwJh7uLW_LJt-yrjnrPlVk-kBrMpBCfyRUruBFjrEL3hci5sQ7ErlKp0km4OT9dVidhy7yIhn5rjspxfljYDSwp20GwsQreznrkqwhe1HJNqS8ig9z-oWLZymBsxnBt4AJ-hHo Jul 19 10:57:58.314860 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Domain-Id: None Jul 19 10:57:58.314860 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Domain-Name: None Jul 19 10:57:58.314860 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Identity-Status: Confirmed Jul 19 10:57:58.314860 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Is-Admin-Project: True Jul 19 10:57:58.314860 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Openstack-Nova-Api-Version: 2.90 Jul 19 10:57:58.314860 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Project-Domain-Id: default Jul 19 10:57:58.314860 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Project-Domain-Name: Default Jul 19 10:57:58.314860 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Project-Id: a1dc1eb7c79345caaeb191fa0f9c008f Jul 19 10:57:58.314860 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Project-Name: admin Jul 19 10:57:58.314860 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Role: member,reader,admin Jul 19 10:57:58.314860 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Roles: member,reader,admin Jul 19 10:57:58.316438 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a1dc1eb7c79345caaeb191fa0f9c008f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/a1dc1eb7c79345caaeb191fa0f9c008f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a1dc1eb7c79345caaeb191fa0f9c008f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_a1dc1eb7c79345caaeb191fa0f9c008f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 10:57:58.316438 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Tenant: admin Jul 19 10:57:58.316438 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Tenant-Id: a1dc1eb7c79345caaeb191fa0f9c008f Jul 19 10:57:58.316438 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-Tenant-Name: admin Jul 19 10:57:58.316438 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-User: admin Jul 19 10:57:58.316438 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-User-Domain-Id: default Jul 19 10:57:58.316438 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-User-Domain-Name: Default Jul 19 10:57:58.316438 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-User-Id: 1eaf5dcf504f41ff99cb8d55073af0f9 Jul 19 10:57:58.316438 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: X-User-Name: admin Jul 19 10:57:58.316438 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: method: > Jul 19 10:57:58.316438 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: exception: Flavor 451 could not be found. Jul 19 10:57:58.316438 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: action_args: {'id': '451'} Jul 19 10:57:58.316438 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 10:57:58.316438 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 10:57:58.317454 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 19 10:57:58.317454 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 19 10:57:58.317454 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 19 10:57:58.317454 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 19 10:57:58.317454 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 19 10:57:58.317454 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 19 10:57:58.317454 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 19 10:57:58.317454 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 19 10:57:58.317454 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 19 10:57:58.317454 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 10:57:58.317454 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 19 10:57:58.317454 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 19 10:57:58.317454 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Jul 19 10:57:58.317454 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi Jul 19 10:57:58.317454 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 10:57:58.317454 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi Jul 19 10:57:58.317454 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 10:57:58.317454 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 10:57:58.317454 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 10:57:58.318689 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 10:57:58.318689 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 10:57:58.318689 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 10:57:58.318689 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 10:57:58.318689 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 19 10:57:58.318689 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 19 10:57:58.318689 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Jul 19 10:57:58.318689 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi Jul 19 10:57:58.318689 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 10:57:58.318689 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi Jul 19 10:57:58.318689 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 10:57:58.318689 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 10:57:58.318689 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 10:57:58.318689 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 10:57:58.318689 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 10:57:58.318689 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Jul 19 10:57:58.318689 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: ERROR nova.api.openstack.wsgi Jul 19 10:57:58.318689 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-380cd15a-8a5c-4d82-9e5d-4713b02f119b admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=103329) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 10:57:58.319891 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-380cd15a-8a5c-4d82-9e5d-4713b02f119b admin admin] 10.222.0.33 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.012247 Jul 19 10:57:58.319891 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 26/52] 10.222.0.33 () {60 vars in 1193 bytes} [Tue Jul 19 10:57:58 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jul 19 10:57:59.273645 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-27d4d50c-2175-439d-833c-84be4bdb2322 admin admin] Calling method '>' {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:57:59.274732 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-27d4d50c-2175-439d-833c-84be4bdb2322 admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001513 Jul 19 10:57:59.275126 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 27/53] 10.222.0.33 () {56 vars in 1074 bytes} [Tue Jul 19 10:57:59 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 10:57:59.788997 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-2d96c090-ce3e-458c-a0df-7a4cc1fcf58e admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 10:57:59.806780 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-2d96c090-ce3e-458c-a0df-7a4cc1fcf58e admin admin] 10.222.0.33 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.90 time: 0.395446 Jul 19 10:57:59.807316 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 27/54] 10.222.0.33 () {64 vars in 1235 bytes} [Tue Jul 19 10:57:59 2022] POST /compute/v2.1/flavors => generated 404 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 19 10:58:00.774993 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-27d4d50c-2175-439d-833c-84be4bdb2322 admin admin] Calling method '>' {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:58:00.776000 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-27d4d50c-2175-439d-833c-84be4bdb2322 admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001431 Jul 19 10:58:00.776393 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 28/55] 10.222.0.33 () {56 vars in 1074 bytes} [Tue Jul 19 10:58:00 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 10:58:01.292662 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-e14d9eeb-ae75-4e9f-9ccc-29bbced89209 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 10:58:01.310046 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-e14d9eeb-ae75-4e9f-9ccc-29bbced89209 admin admin] 10.222.0.33 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.90 time: 0.397757 Jul 19 10:58:01.310586 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 28/56] 10.222.0.33 () {64 vars in 1235 bytes} [Tue Jul 19 10:58:00 2022] POST /compute/v2.1/flavors => generated 405 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 19 10:58:02.274243 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-27d4d50c-2175-439d-833c-84be4bdb2322 admin admin] Calling method '>' {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:58:02.275246 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-27d4d50c-2175-439d-833c-84be4bdb2322 admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001419 Jul 19 10:58:02.275639 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 29/57] 10.222.0.33 () {56 vars in 1074 bytes} [Tue Jul 19 10:58:02 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 10:58:02.795914 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-c7a38e99-7262-43b0-82df-dd54f7982be7 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 10:58:02.813998 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-c7a38e99-7262-43b0-82df-dd54f7982be7 admin admin] 10.222.0.33 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.90 time: 0.403491 Jul 19 10:58:02.814539 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 29/58] 10.222.0.33 () {64 vars in 1235 bytes} [Tue Jul 19 10:58:02 2022] POST /compute/v2.1/flavors => generated 407 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 19 10:58:03.779073 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-27d4d50c-2175-439d-833c-84be4bdb2322 admin admin] Calling method '>' {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:58:03.780199 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-27d4d50c-2175-439d-833c-84be4bdb2322 admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001545 Jul 19 10:58:03.780591 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 30/59] 10.222.0.33 () {56 vars in 1074 bytes} [Tue Jul 19 10:58:03 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 10:58:04.316598 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-5c00abfa-2455-4706-9076-cbebb56cd80f admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 10:58:04.336625 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-5c00abfa-2455-4706-9076-cbebb56cd80f admin admin] 10.222.0.33 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.90 time: 0.410632 Jul 19 10:58:04.337195 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 30/60] 10.222.0.33 () {64 vars in 1235 bytes} [Tue Jul 19 10:58:03 2022] POST /compute/v2.1/flavors => generated 409 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 19 10:58:05.306056 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-27d4d50c-2175-439d-833c-84be4bdb2322 admin admin] Calling method '>' {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:58:05.307059 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-27d4d50c-2175-439d-833c-84be4bdb2322 admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001428 Jul 19 10:58:05.307449 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 31/61] 10.222.0.33 () {56 vars in 1074 bytes} [Tue Jul 19 10:58:05 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 10:58:05.824288 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-fe92d675-3d93-407c-ab87-392577f73228 admin admin] Calling method '>' {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:58:05.843392 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-fe92d675-3d93-407c-ab87-392577f73228 admin admin] 10.222.0.33 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.90 time: 0.398823 Jul 19 10:58:05.843916 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 31/62] 10.222.0.33 () {60 vars in 1202 bytes} [Tue Jul 19 10:58:05 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 10:58:57.303281 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-27d4d50c-2175-439d-833c-84be4bdb2322 admin admin] Calling method '>' {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:58:57.304332 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-27d4d50c-2175-439d-833c-84be4bdb2322 admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001444 Jul 19 10:58:57.304741 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 32/63] 10.222.0.33 () {54 vars in 930 bytes} [Tue Jul 19 10:58:57 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 10:58:57.315105 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG dbcounter [-] [103329] Writing DB stats nova_api:SELECT=26,nova_api:INSERT=4 {{(pid=103329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 10:58:57.693624 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-6fa29bbb-421c-4ed3-9bc5-953a37df5bfd admin admin] Calling method '>' {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:58:57.712932 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-6fa29bbb-421c-4ed3-9bc5-953a37df5bfd admin admin] 10.222.0.33 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.402255 Jul 19 10:58:57.713486 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 32/64] 10.222.0.33 () {60 vars in 1291 bytes} [Tue Jul 19 10:58:57 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 10:58:57.735944 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-9b08cf1c-0b96-48f1-8501-4ccbaf1a2928 admin admin] Calling method '>' {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:58:57.739457 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG dbcounter [-] [103328] Writing DB stats nova_api:SELECT=14 {{(pid=103328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 10:58:57.745954 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-9b08cf1c-0b96-48f1-8501-4ccbaf1a2928 admin admin] 10.222.0.33 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.013604 Jul 19 10:58:57.746476 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 33/65] 10.222.0.33 () {60 vars in 1282 bytes} [Tue Jul 19 10:58:57 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 15 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 10:58:57.757914 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-9fb4f68a-0ceb-493d-afc4-427ee92c09af admin admin] Calling method '>' {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:58:57.765934 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-9fb4f68a-0ceb-493d-afc4-427ee92c09af admin admin] 10.222.0.33 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011530 Jul 19 10:58:57.766442 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 33/66] 10.222.0.33 () {60 vars in 1285 bytes} [Tue Jul 19 10:58:57 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 10:58:57.777324 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-386ccab5-d268-4879-ac5c-1d3b9038bc0f admin admin] Calling method '>' {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:58:57.784677 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-386ccab5-d268-4879-ac5c-1d3b9038bc0f admin admin] 10.222.0.33 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.010789 Jul 19 10:58:57.785207 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 34/67] 10.222.0.33 () {60 vars in 1285 bytes} [Tue Jul 19 10:58:57 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 10:58:57.796062 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-b1de59bc-10d4-4105-8406-2e53d54e7c75 admin admin] Calling method '>' {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 10:58:57.803396 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-b1de59bc-10d4-4105-8406-2e53d54e7c75 admin admin] 10.222.0.33 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.010737 Jul 19 10:58:57.803905 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 34/68] 10.222.0.33 () {60 vars in 1282 bytes} [Tue Jul 19 10:58:57 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:02:22.201342 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-386ccab5-d268-4879-ac5c-1d3b9038bc0f admin admin] Calling method '>' {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:02:22.202393 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-386ccab5-d268-4879-ac5c-1d3b9038bc0f admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001477 Jul 19 11:02:22.202783 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 35/69] 10.222.0.33 () {56 vars in 1074 bytes} [Tue Jul 19 11:02:22 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:02:22.343609 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG dbcounter [-] [103329] Writing DB stats nova_api:SELECT=6 {{(pid=103329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:02:22.498432 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-e7c33703-2719-4e36-b41b-b43e79b34182 admin admin] Calling method '>' {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:02:22.499664 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 19 11:02:22.499664 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: warnings.warn(msg) Jul 19 11:02:22.500896 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG oslo_concurrency.lockutils [None req-e7c33703-2719-4e36-b41b-b43e79b34182 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103329) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:02:22.501339 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG oslo_concurrency.lockutils [None req-e7c33703-2719-4e36-b41b-b43e79b34182 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=103329) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:02:22.502121 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG oslo_concurrency.lockutils [None req-e7c33703-2719-4e36-b41b-b43e79b34182 admin admin] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103329) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:02:22.502549 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG oslo_concurrency.lockutils [None req-e7c33703-2719-4e36-b41b-b43e79b34182 admin admin] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103329) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:02:22.536580 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-e7c33703-2719-4e36-b41b-b43e79b34182 admin admin] 10.222.0.33 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.90 time: 0.194379 Jul 19 11:02:22.537163 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 35/70] 10.222.0.33 () {60 vars in 1193 bytes} [Tue Jul 19 11:02:22 2022] GET /compute/v2.1/os-services => generated 791 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 19 11:02:23.359763 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-386ccab5-d268-4879-ac5c-1d3b9038bc0f admin admin] Calling method '>' {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:02:23.360802 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-386ccab5-d268-4879-ac5c-1d3b9038bc0f admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001411 Jul 19 11:02:23.361201 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 36/71] 10.222.0.33 () {56 vars in 1074 bytes} [Tue Jul 19 11:02:23 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:02:23.549385 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-140713fa-9468-4ac4-8145-f00c15fab368 admin admin] Calling method '>' {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:02:23.550567 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 19 11:02:23.550567 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: warnings.warn(msg) Jul 19 11:02:23.551752 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG oslo_concurrency.lockutils [None req-140713fa-9468-4ac4-8145-f00c15fab368 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103329) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:02:23.552194 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG oslo_concurrency.lockutils [None req-140713fa-9468-4ac4-8145-f00c15fab368 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103329) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:02:23.553024 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG oslo_concurrency.lockutils [None req-140713fa-9468-4ac4-8145-f00c15fab368 admin admin] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103329) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:02:23.553460 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG oslo_concurrency.lockutils [None req-140713fa-9468-4ac4-8145-f00c15fab368 admin admin] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103329) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:02:23.572744 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-140713fa-9468-4ac4-8145-f00c15fab368 admin admin] 10.222.0.33 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.90 time: 0.075204 Jul 19 11:02:23.573292 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 36/72] 10.222.0.33 () {60 vars in 1193 bytes} [Tue Jul 19 11:02:23 2022] GET /compute/v2.1/os-services => generated 791 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 19 11:02:34.789143 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-386ccab5-d268-4879-ac5c-1d3b9038bc0f admin admin] Calling method '>' {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:02:34.790149 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-386ccab5-d268-4879-ac5c-1d3b9038bc0f admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001464 Jul 19 11:02:34.790544 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 37/73] 10.222.0.33 () {56 vars in 1074 bytes} [Tue Jul 19 11:02:34 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:02:34.935742 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG dbcounter [-] [103329] Writing DB stats nova_cell0:SELECT=4 {{(pid=103329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:02:34.937003 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG dbcounter [-] [103329] Writing DB stats nova_cell1:SELECT=4 {{(pid=103329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:02:34.937842 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG dbcounter [-] [103329] Writing DB stats nova_api:SELECT=8 {{(pid=103329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:02:34.986137 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-1e86a6b9-1138-485c-be6a-8fb23069342c admin admin] Calling method '>' {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:02:34.987409 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 19 11:02:34.987409 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: warnings.warn(msg) Jul 19 11:02:34.988599 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG oslo_concurrency.lockutils [None req-1e86a6b9-1138-485c-be6a-8fb23069342c admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103329) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:02:34.989079 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG oslo_concurrency.lockutils [None req-1e86a6b9-1138-485c-be6a-8fb23069342c admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103329) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:02:34.989864 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG oslo_concurrency.lockutils [None req-1e86a6b9-1138-485c-be6a-8fb23069342c admin admin] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103329) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:02:34.990296 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG oslo_concurrency.lockutils [None req-1e86a6b9-1138-485c-be6a-8fb23069342c admin admin] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103329) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:02:35.015234 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-1e86a6b9-1138-485c-be6a-8fb23069342c admin admin] 10.222.0.33 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.90 time: 0.084443 Jul 19 11:02:35.015774 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 37/74] 10.222.0.33 () {60 vars in 1193 bytes} [Tue Jul 19 11:02:34 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:02:35.838761 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: DEBUG nova.api.openstack.wsgi [None req-386ccab5-d268-4879-ac5c-1d3b9038bc0f admin admin] Calling method '>' {{(pid=103328) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:02:35.839757 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: INFO nova.api.openstack.requestlog [None req-386ccab5-d268-4879-ac5c-1d3b9038bc0f admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002035 Jul 19 11:02:35.840156 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: [pid: 103328|app: 0|req: 38/75] 10.222.0.33 () {56 vars in 1074 bytes} [Tue Jul 19 11:02:35 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:02:36.035323 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG nova.api.openstack.wsgi [None req-292ebdbc-6394-4292-9d5e-ca01ae9dc5fe admin admin] Calling method '>' {{(pid=103329) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:02:36.036479 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 19 11:02:36.036479 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: warnings.warn(msg) Jul 19 11:02:36.037685 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG oslo_concurrency.lockutils [None req-292ebdbc-6394-4292-9d5e-ca01ae9dc5fe admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103329) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:02:36.038118 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG oslo_concurrency.lockutils [None req-292ebdbc-6394-4292-9d5e-ca01ae9dc5fe admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103329) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:02:36.038888 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG oslo_concurrency.lockutils [None req-292ebdbc-6394-4292-9d5e-ca01ae9dc5fe admin admin] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103329) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:02:36.039337 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: DEBUG oslo_concurrency.lockutils [None req-292ebdbc-6394-4292-9d5e-ca01ae9dc5fe admin admin] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103329) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:02:36.060965 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: INFO nova.api.openstack.requestlog [None req-292ebdbc-6394-4292-9d5e-ca01ae9dc5fe admin admin] 10.222.0.33 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.90 time: 0.077591 Jul 19 11:02:36.061451 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: [pid: 103329|app: 0|req: 38/76] 10.222.0.33 () {60 vars in 1193 bytes} [Tue Jul 19 11:02:35 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:03:34.469663 dv-cinder-master-3-iscsi devstack@n-api.service[103327]: Tue Jul 19 11:03:34 2022 - graceful shutdown triggered... Jul 19 11:03:34.470165 dv-cinder-master-3-iscsi devstack@n-api.service[103328]: Gracefully killing worker 1 (pid: 103328)... Jul 19 11:03:34.470545 dv-cinder-master-3-iscsi devstack@n-api.service[103329]: Gracefully killing worker 2 (pid: 103329)... Jul 19 11:03:34.472462 dv-cinder-master-3-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Jul 19 11:03:35.471264 dv-cinder-master-3-iscsi devstack@n-api.service[103327]: worker 1 buried after 1 seconds Jul 19 11:03:35.471795 dv-cinder-master-3-iscsi devstack@n-api.service[103327]: worker 2 buried after 1 seconds Jul 19 11:03:35.471795 dv-cinder-master-3-iscsi devstack@n-api.service[103327]: goodbye to uWSGI. Jul 19 11:03:35.472821 dv-cinder-master-3-iscsi systemd[1]: devstack@n-api.service: Succeeded. Jul 19 11:03:35.473361 dv-cinder-master-3-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Jul 19 11:03:35.476102 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Jul 19 11:03:35.485773 dv-cinder-master-3-iscsi devstack@n-api.service[117832]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Jul 19 11:03:35.487664 dv-cinder-master-3-iscsi devstack@n-api.service[117832]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jul 19 11:03:35 2022] *** Jul 19 11:03:35.487664 dv-cinder-master-3-iscsi devstack@n-api.service[117832]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 19 11:03:35.487664 dv-cinder-master-3-iscsi devstack@n-api.service[117832]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 19 11:03:35.487664 dv-cinder-master-3-iscsi devstack@n-api.service[117832]: nodename: dv-cinder-master-3-iscsi Jul 19 11:03:35.487664 dv-cinder-master-3-iscsi devstack@n-api.service[117832]: machine: x86_64 Jul 19 11:03:35.487664 dv-cinder-master-3-iscsi devstack@n-api.service[117832]: clock source: unix Jul 19 11:03:35.487664 dv-cinder-master-3-iscsi devstack@n-api.service[117832]: pcre jit disabled Jul 19 11:03:35.487664 dv-cinder-master-3-iscsi devstack@n-api.service[117832]: detected number of CPU cores: 4 Jul 19 11:03:35.487664 dv-cinder-master-3-iscsi devstack@n-api.service[117832]: current working directory: / Jul 19 11:03:35.487664 dv-cinder-master-3-iscsi devstack@n-api.service[117832]: detected binary path: /usr/bin/uwsgi-core Jul 19 11:03:35.488318 dv-cinder-master-3-iscsi devstack@n-api.service[117832]: your processes number limit is 47680 Jul 19 11:03:35.488318 dv-cinder-master-3-iscsi devstack@n-api.service[117832]: your memory page size is 4096 bytes Jul 19 11:03:35.488318 dv-cinder-master-3-iscsi devstack@n-api.service[117832]: detected max file descriptor number: 2048 Jul 19 11:03:35.488318 dv-cinder-master-3-iscsi devstack@n-api.service[117832]: lock engine: pthread robust mutexes Jul 19 11:03:35.488318 dv-cinder-master-3-iscsi devstack@n-api.service[117832]: thunder lock: enabled Jul 19 11:03:35.488318 dv-cinder-master-3-iscsi devstack@n-api.service[117832]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Jul 19 11:03:35.488318 dv-cinder-master-3-iscsi devstack@n-api.service[117832]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 19 11:03:35.532852 dv-cinder-master-3-iscsi devstack@n-api.service[117832]: Python main interpreter initialized at 0x560fd77459c0 Jul 19 11:03:35.532852 dv-cinder-master-3-iscsi devstack@n-api.service[117832]: python threads support enabled Jul 19 11:03:35.532852 dv-cinder-master-3-iscsi devstack@n-api.service[117832]: your server socket listen backlog is limited to 100 connections Jul 19 11:03:35.532852 dv-cinder-master-3-iscsi devstack@n-api.service[117832]: your mercy for graceful operations on workers is 90 seconds Jul 19 11:03:35.533264 dv-cinder-master-3-iscsi devstack@n-api.service[117832]: mapped 403077 bytes (393 KB) for 2 cores Jul 19 11:03:35.533332 dv-cinder-master-3-iscsi devstack@n-api.service[117832]: *** Operational MODE: preforking *** Jul 19 11:03:35.533594 dv-cinder-master-3-iscsi devstack@n-api.service[117832]: *** uWSGI is running in multiple interpreter mode *** Jul 19 11:03:35.533594 dv-cinder-master-3-iscsi devstack@n-api.service[117832]: spawned uWSGI master process (pid: 117832) Jul 19 11:03:35.535783 dv-cinder-master-3-iscsi devstack@n-api.service[117832]: spawned uWSGI worker 1 (pid: 117848, cores: 1) Jul 19 11:03:35.535783 dv-cinder-master-3-iscsi devstack@n-api.service[117832]: spawned uWSGI worker 2 (pid: 117849, cores: 1) Jul 19 11:03:35.535783 dv-cinder-master-3-iscsi devstack@n-api.service[117832]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 19 11:03:35.533780 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Jul 19 11:03:36.190148 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Jul 19 11:03:36.195083 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Jul 19 11:03:37.382408 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 19 11:03:37.382408 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: from cryptography.utils import int_from_bytes Jul 19 11:03:37.383538 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 19 11:03:37.383538 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: from cryptography.utils import int_from_bytes Jul 19 11:03:37.387035 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 19 11:03:37.387035 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: from cryptography.utils import int_from_bytes Jul 19 11:03:37.387491 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 19 11:03:37.387491 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: from cryptography.utils import int_from_bytes Jul 19 11:03:39.696268 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 19 11:03:39.697083 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 19 11:03:39.697469 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 19 11:03:39.697594 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 19 11:03:39.697688 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 19 11:03:39.697773 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 19 11:03:39.697868 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 19 11:03:39.697952 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 19 11:03:39.698051 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 19 11:03:39.698160 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 19 11:03:39.698392 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.698502 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.698665 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.698805 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.699007 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.699168 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.699514 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.699613 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.699723 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.699839 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.699937 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.700135 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.700253 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.700499 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.700641 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.700864 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.700965 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.701128 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.701283 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.701443 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.701599 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.701768 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.701898 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.702092 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.702225 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.702404 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.702609 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.702774 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.702918 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.703056 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.703443 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.703542 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.703630 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.703709 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.703893 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.704014 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.704292 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.704404 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.704608 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.704932 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.705277 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.705420 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.705665 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.705944 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.712811 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.713234 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.713363 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.713464 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.713566 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.713680 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.713879 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.714022 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.714144 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.714241 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.714585 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.714742 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.714742 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.714882 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.714960 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.715056 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.715152 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.715246 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.715339 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.715440 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.715525 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.715607 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.715697 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.715780 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.715859 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.715859 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.716017 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.716095 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.716166 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.716235 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.716312 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.716395 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.716395 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.716540 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.716612 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.716713 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.716791 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.716880 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.716963 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.717046 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.717123 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.717203 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.717275 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.717358 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.717429 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.717499 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.717575 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.717652 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.717735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.717806 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.717875 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.717951 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.718028 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.718105 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.718209 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.718281 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.718374 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.718457 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.718529 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.718605 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.718685 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.718765 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.718837 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.718910 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.718987 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.719067 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.719148 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.719226 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.719299 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.719376 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.719482 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.719566 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.719566 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.719704 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.719773 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.719843 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.719920 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.719997 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.720095 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.720175 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.720247 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.720317 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.720399 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.721293 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.721940 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.722111 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.722267 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.722420 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.722583 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.722730 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.722886 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.723040 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.723200 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.723394 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.723592 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.723743 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.723911 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.723911 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.724370 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.724533 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.724717 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.33 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.724925 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.33 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.725079 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.33 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.725234 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.33 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.725377 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.725534 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.725679 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.725824 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.725975 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.726124 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.726277 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.726435 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.726579 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.726723 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.726723 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.727004 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.727172 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.727316 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.727466 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.727615 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.727767 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.727918 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.728077 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.728233 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.728389 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.728540 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.728709 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.728863 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.729022 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.729183 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.729344 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.729492 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.729635 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.729783 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.729941 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.729941 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.730282 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.730426 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.730577 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.730738 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.730892 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.731056 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.731206 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.731363 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.731512 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.731944 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.732110 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.732263 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.732413 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.732565 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.732730 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.732898 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.733052 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.733204 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.733366 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.733517 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.733668 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.733893 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.734049 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.734193 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.734335 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.734486 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.734637 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.734792 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.734935 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.735079 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.735222 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.735383 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.735536 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.735681 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.735824 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.735977 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.736121 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.736272 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.736442 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.737224 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.737383 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.737528 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.737671 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.737823 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.737989 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.738141 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.738294 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.738447 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.738599 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.738752 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.738910 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.739063 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.739213 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.739369 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.739520 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.739671 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.739825 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.739998 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.740153 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.740299 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.740443 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.740593 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.740808 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.740959 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.741110 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.741265 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.741417 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.741572 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.741741 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.742136 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.742793 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.742941 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.743085 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.743275 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.743426 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.743572 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.743723 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.743868 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.744011 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.744166 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.744318 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.744462 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.744617 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.744786 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.744942 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.745096 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.745247 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.745399 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.745551 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.745703 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.745867 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.746282 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.746428 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.746580 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.746733 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 11:03:39.746895 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.747049 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.747203 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.747347 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.747499 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.747651 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.747795 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.747954 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.748099 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.748249 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.748395 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.748537 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.748752 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.748914 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.749059 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.749202 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.749353 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.749510 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.749664 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.749820 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.749975 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.750130 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.750282 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.750434 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.750588 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.750745 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.750890 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.751041 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.751205 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.751358 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.751510 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.751663 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.751815 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.751978 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.752845 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.752845 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.753031 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.753128 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.753213 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.753213 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.753373 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.753467 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.753548 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.753548 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.753685 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.753685 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.753857 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.753940 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.754027 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.754120 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.754217 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.754217 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.754373 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.754444 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.754523 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.754601 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.754672 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.754740 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.754815 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.754893 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.754893 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.754893 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.755104 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.755182 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.755259 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.755336 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.755420 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.755498 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.755576 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.755646 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.755646 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.755798 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.755872 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.755941 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.756029 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.756101 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.756318 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.756455 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.756653 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.757006 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.757415 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.757520 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.757635 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.757782 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.758514 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.758800 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.758907 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.759120 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.759248 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.759428 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.759730 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.759894 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.760038 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.760349 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.760651 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.760975 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.761104 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.761298 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.761403 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.761617 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.761731 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.761940 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.762048 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.762242 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.762330 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.762559 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.762673 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.762862 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.762988 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.763169 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.763260 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.763474 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.763581 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.763781 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.763880 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.764082 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.764173 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.764392 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.764484 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.764765 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.764874 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.765030 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.765122 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.765350 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.765452 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.765656 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.765755 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.765958 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.766057 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.766265 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.766404 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.766565 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.766665 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.766902 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.767043 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.767207 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.767320 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.767472 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.767569 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.767777 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.767880 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.768085 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.768273 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.768375 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.768466 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.768723 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.768840 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.769026 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.769133 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.769332 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.769422 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.769647 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.769749 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.769958 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.770051 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.770268 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.770393 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.770576 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.770676 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.770890 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.771000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.771201 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.771298 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.771502 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.771594 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.771806 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.771905 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.772108 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.772198 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.772408 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.772506 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.772721 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.772815 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.773031 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.773182 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.773332 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.773432 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.773657 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.773755 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.773957 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.774056 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.774259 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.774357 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.774556 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.774648 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.774874 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.774975 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.775185 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.775283 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.775486 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.775590 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.775785 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.775884 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.776089 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.776177 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.776398 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.776495 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.776718 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.776827 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.777031 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.777133 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.777355 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.777457 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.777667 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.777767 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.777972 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.778064 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.778277 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.778375 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.778584 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.778683 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.778889 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.778986 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.779199 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.779289 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.779508 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.779599 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.779817 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.779911 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.780133 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.780225 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.780442 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.780536 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.780789 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.780894 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.781099 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.781197 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.781419 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.781510 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.781728 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.781864 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.782040 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.782163 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.782341 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.782430 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.782652 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.782745 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.782963 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.783081 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.783360 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.783497 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.783610 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.783720 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.783936 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.784065 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.784246 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.784348 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.784561 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.784690 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.784897 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.785076 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.785244 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.785340 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.788720 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.788849 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.788963 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.789068 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.789180 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.789254 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.789334 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.789405 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.789485 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.789556 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.789625 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.789702 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.789780 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.789857 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.789942 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.790013 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.790081 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.790157 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.790234 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.790311 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.790466 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.790547 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.790547 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.790547 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.790547 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.790547 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.790547 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.790547 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.790992 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.791167 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.791417 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.791512 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.791795 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.791892 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.792108 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.792210 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.792421 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.792546 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.792771 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.792871 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.793076 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.793183 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.793390 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.793500 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.793702 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.793798 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.794009 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.794111 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.794424 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.794534 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.794610 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.794681 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.794921 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.795024 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.795229 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.795360 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.795538 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.795658 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.795845 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.795962 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.796157 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.796404 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.796547 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.796667 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.796823 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.796961 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.797102 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.797430 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.797699 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.797828 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.797954 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.798064 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.798188 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.798299 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.798419 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.798536 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.798654 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.798764 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.798894 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.799013 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.799166 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.799286 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.799487 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.799622 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.799794 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.799912 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.800096 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.800820 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.800820 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.801067 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.801239 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.801408 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.801555 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.801709 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.801855 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.802009 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.802156 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.802309 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.802463 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.802608 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.802771 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.803009 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.803171 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.803323 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.803488 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.803633 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.803791 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.803945 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.804091 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.804242 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.804407 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.805264 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.805444 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.805591 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.805735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.805880 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.806030 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.806185 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.806346 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.806494 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.806657 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.807521 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.807834 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.807997 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.808170 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.808343 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.808500 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.808646 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.808827 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.809001 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.809156 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.809301 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.809453 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.809599 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.809750 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.809904 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.810058 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.810203 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.810203 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.810489 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.810489 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.810871 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.811040 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.811186 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.811338 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.811484 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.811640 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.811794 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.811947 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.812093 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.812246 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.812435 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.812435 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.812791 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.812945 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.813206 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.813378 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.813538 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.813685 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.813877 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.814034 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.814188 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.814333 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.814485 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.814659 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.814919 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.815160 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.815330 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.815477 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.815631 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.815785 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.815950 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.816096 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.816256 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.816403 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.816547 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.816719 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.816888 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.817036 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.817181 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.817336 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.817686 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.817994 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.818155 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.818310 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.818455 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.818600 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.818752 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.818907 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.819054 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.819199 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.819351 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.819504 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.819661 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.819819 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.819965 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.820111 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.820256 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.820461 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.821198 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.821362 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.821517 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.821663 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.821807 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.821951 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.822180 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.822347 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.822492 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.822638 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.822782 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.822934 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.823089 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.823338 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.823498 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.823644 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.823796 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.823950 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.824111 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.824278 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.824437 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.824596 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.824789 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.825062 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.825326 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.825495 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.825648 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.825799 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.825950 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.826114 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.826276 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.826428 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.826600 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.826753 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.826911 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.827077 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.827238 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.827392 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.827565 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.828040 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.828213 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.828851 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.829024 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.829178 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.829337 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.829509 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.829675 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.829836 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.829995 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.830147 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.830304 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.830464 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.830702 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.830865 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.831017 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.831198 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.831420 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.831579 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.831756 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.831909 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.832060 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.832220 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.832371 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.832529 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.832714 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.832885 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.833045 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.833197 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.833456 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.833625 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.833803 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.834095 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.834253 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.834441 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.834597 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.834767 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.834927 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.835096 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.835359 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.835626 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.835794 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.835987 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.836147 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.836320 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.836472 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.836658 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.836867 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.837034 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.837196 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.837368 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.837529 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.837725 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.838224 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.838385 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.838537 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.838689 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.838854 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.839006 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.839006 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.839304 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.839456 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.839614 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.839774 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.839927 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.840087 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.840247 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.840407 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.840575 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.840763 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.842009 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.842190 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.842345 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.842497 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.842657 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.842809 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.842970 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.843124 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.843274 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.843457 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.843695 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.843860 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.844032 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.844291 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.844448 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.844608 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.844805 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.844973 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.845137 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.845298 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.845556 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.845820 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.845978 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.846130 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.846297 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.846461 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.846612 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.846762 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.846927 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.847093 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.847262 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.847415 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.847573 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.847725 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.847875 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.848373 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.848543 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.848720 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.848917 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.849072 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.849223 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.849388 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.849541 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.849692 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.849842 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.850007 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.850161 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.850326 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.850488 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.850639 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.850791 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.850952 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.851112 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.851273 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.851424 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.851610 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.851778 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.851938 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.852099 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.852258 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.852417 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.852570 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.852749 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.852907 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.853091 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.853245 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.853395 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.853547 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.853715 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.853944 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.854110 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.854290 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.854510 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.854667 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.854827 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.854979 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.855139 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.855303 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.855465 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.855720 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.855974 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.856149 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.856318 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.856470 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.856629 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.856819 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.856819 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.857133 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.857133 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.857450 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.857610 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.857770 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.857770 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.858099 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.858569 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.858738 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.858902 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.859054 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.859206 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.859368 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.859530 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.859681 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.859681 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.859981 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.860143 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.860295 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.860295 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.860589 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.860760 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.860930 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.861092 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.861254 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.861414 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.861573 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.861731 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.861893 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.862056 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.862208 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.862359 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.862509 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.862509 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.862801 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.862966 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.863119 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.863119 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.863415 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.863575 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.863815 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.864077 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.864246 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.864407 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.864641 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.864820 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.864991 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.865153 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.865305 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.865464 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.865615 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.865873 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.866405 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.866579 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.866740 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.866892 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.867052 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.867714 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.867891 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.868055 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.868207 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.868207 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.868938 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.869104 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.869265 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.869265 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.869565 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.869717 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.869876 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.870039 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.870191 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.870351 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.870502 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.870663 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.870822 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.871238 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.871401 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.871561 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.871720 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.871883 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.872034 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.872198 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.872361 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.872529 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.872689 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.872925 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.873099 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.873261 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.873413 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.873564 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.873723 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.873899 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.874079 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.874310 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.874476 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.874649 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.874883 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.875041 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.875193 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.875344 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.875495 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.875649 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.875809 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.876080 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.876349 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.876515 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.876679 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.876855 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.877024 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.877178 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.877336 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.877488 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.877639 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.877800 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.877966 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.878119 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.878278 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.878472 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.878957 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.879133 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.879295 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.879448 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.879607 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.879768 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.879928 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.880095 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.880248 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.880408 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.880560 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.880765 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.880936 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.881101 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.881253 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.881404 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.881555 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.881714 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.881874 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.882041 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.882193 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.882343 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.882493 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.882644 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.882819 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.883542 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.883700 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.883852 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.884012 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.884171 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.884349 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.884585 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.884759 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.885000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.885158 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.885321 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.885484 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.885648 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.885648 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.885946 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.886115 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.886381 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.886381 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.886785 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.886946 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.887106 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.887259 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.887421 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.887581 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.887734 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.887904 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.888056 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.888209 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.888369 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.888520 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.888679 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.889199 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.889372 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.889525 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.889759 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.889759 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.890058 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.890231 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.890393 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.890545 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.890706 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.890857 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.891015 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.891167 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.891319 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.891484 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.891644 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.891796 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.891947 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.892106 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.892277 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.892445 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.892605 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.892781 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.892940 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.893091 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.893253 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.893416 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.893576 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.893734 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.893892 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.894043 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.894203 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.894375 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.894536 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.894785 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.894950 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.895200 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.895371 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.895532 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.895693 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.895845 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.896008 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.896161 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.896578 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.896849 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.897021 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.897235 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.897401 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.897561 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.897724 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.897878 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.898037 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.898198 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.898348 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.898538 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.898708 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.898870 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.899375 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.899546 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.899700 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.899866 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.900026 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.900187 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.900347 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.900508 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.900660 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.900891 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.901047 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.901199 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.901350 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.901513 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.901671 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.901839 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.901992 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.902144 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.902302 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.902456 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.902607 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.902778 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.902933 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.903084 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.903251 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.903251 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.903565 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.903718 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.903870 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.903870 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.904166 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.904326 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.904489 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.904642 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.904909 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.905070 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.905070 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.905467 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.905642 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.905804 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.905967 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.906121 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.906273 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.906432 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.906617 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.907046 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.907229 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.907385 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.907536 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.907698 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.907698 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.908000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.908000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.908305 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.908458 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.908622 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.908799 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.908967 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.909141 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.909657 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.909823 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.909981 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.910145 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.910298 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.910449 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.910601 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.910763 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.910915 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.911067 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.911220 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.911384 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.911546 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.911700 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.911860 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.912028 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.912187 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.912380 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.912548 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.912786 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.912948 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.913112 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.913274 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.913436 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.913599 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.913752 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.913910 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.914063 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.914267 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.914436 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.914597 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.914750 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.914919 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.915160 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.915343 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.915584 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.915742 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.915910 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.916075 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.916229 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.916391 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.916391 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.916710 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.916889 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.917236 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.917404 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.917568 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.917722 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.917894 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.917894 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.918183 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.918335 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.918511 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.918673 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.918824 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.918976 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.919127 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.919296 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.919788 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.919950 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.920102 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.920253 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.920412 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.920573 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.920806 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.920979 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.921138 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.924650 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.924842 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.925087 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.925087 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.925524 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.925777 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.925953 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.926110 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.926278 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.926278 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.926569 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.926734 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.926893 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.927046 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.927413 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.927576 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.927738 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.927890 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.928042 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.928203 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.928385 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.928385 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.936343 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.937161 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.937346 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.937728 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.937912 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.938131 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.938291 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.938454 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.938644 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.939321 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.939487 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.939642 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.939795 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.939957 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.940137 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.940291 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.940442 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.940603 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.940802 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.940962 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.941128 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.941282 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.941433 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.941593 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.941756 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.941918 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.942082 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.942243 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.942395 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.942560 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.942723 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.942885 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.943037 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.944536 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.944743 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.944911 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.945077 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.945244 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.945990 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.946516 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.946687 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.946849 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.947003 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.947186 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.947348 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.947500 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.947660 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.947821 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.947988 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.948140 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.948312 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.948569 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.948842 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.949004 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.949170 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.949322 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.949481 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.949643 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.949804 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.949965 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.950134 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.950295 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.950447 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.950598 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.950756 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.950926 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.951094 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.951247 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.951406 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.951559 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.951718 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.951880 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.952039 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.952200 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.952360 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.952523 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.952683 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.952878 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.953056 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.953056 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.953347 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.953506 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.953659 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.953819 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.953978 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.954281 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.954450 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.954610 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.954770 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.954770 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.955096 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.955251 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.955403 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.955555 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.955720 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.955880 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.956033 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.956183 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.956183 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.956884 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.957051 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.957204 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.957365 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.957528 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.957687 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.957839 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.958001 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.958153 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.958313 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.958467 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.958723 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.958980 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.959151 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.959304 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.959459 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.959631 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.959783 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.959945 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.960099 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.960250 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.960402 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.960562 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.960736 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.960915 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.961256 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.961411 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.961565 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.961732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.961899 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.962060 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.962221 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.962374 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.962526 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.962689 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.962852 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.963005 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.963165 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.963326 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.963487 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.963650 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.963830 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.963985 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.964145 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.964551 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.964788 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.964951 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.965123 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.965277 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.965430 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.965590 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.965753 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.965916 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.966069 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.966240 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.966425 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.967459 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.967649 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.967814 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.967982 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.968144 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.968299 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.968452 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.968612 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.969072 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.969379 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.969555 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.969718 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.969880 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.970050 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.970211 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.970365 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.970518 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.970670 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.970670 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.970670 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.970670 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.971266 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.971434 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.971434 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.971724 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.971892 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.972046 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.972198 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.972359 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.972512 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.972678 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.972678 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.973233 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.973233 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.973559 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.973736 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.973903 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.974057 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.974210 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.974363 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.974776 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.974938 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.975105 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.975261 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.975413 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.975573 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.975732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.975910 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.976073 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.976228 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.976389 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.976561 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.977534 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.977713 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.977868 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.978021 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.978173 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.978327 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.978480 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.978660 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.978830 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.979006 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.979308 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.979631 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.979631 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.979940 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.980093 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.980245 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.980396 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.980549 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.980549 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.980885 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.981049 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.981231 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.981384 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.981545 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.981712 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.981875 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.982085 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.982253 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.982415 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.982570 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.982730 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.982883 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.983042 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.983204 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.983375 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.983528 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.983528 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.983838 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.984006 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.984159 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.984310 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.984461 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.984673 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.985118 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.985273 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.985435 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.985602 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.985766 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.985918 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.986070 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.986228 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.986382 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.986534 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.986703 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.986870 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.987625 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.988402 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.988580 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.988757 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.988936 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.989100 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.989840 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.990016 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.990181 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.990344 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.990506 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.990668 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.990863 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.991623 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.991807 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.991984 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.992145 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.992925 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.993097 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.993251 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.993405 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.993558 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.993710 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.993872 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.994034 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.994188 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.994340 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.994519 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.994681 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.994844 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.995280 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.995453 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.995453 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.995744 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.995898 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.996478 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.996664 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.996846 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.996999 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.997205 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.997958 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.998125 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.998280 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.998431 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.998585 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.998744 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.998744 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.999047 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.999201 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.999355 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.999507 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:39.999661 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:40.000039 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:40.000202 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:40.000356 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:40.000520 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:40.000869 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:40.001046 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:40.001207 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:40.001361 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:40.001361 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=117848) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 19 11:03:40.001672 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:40.001672 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:40.001672 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:40.001672 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:40.001672 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:40.001672 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:40.001672 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:40.001672 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:40.002824 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:40.002824 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:40.002824 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:40.002824 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 11:03:40.002824 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=117849) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 19 11:03:40.111796 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO dbcounter [None req-dcbfff08-c38c-4dc2-aeb4-4ed79589ef4a None None] Registered counter for database nova_api Jul 19 11:03:40.116448 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO dbcounter [None req-133cd534-c08f-46a7-b241-c32db1c6184b None None] Registered counter for database nova_api Jul 19 11:03:40.133713 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_db.sqlalchemy.engines [None req-dcbfff08-c38c-4dc2-aeb4-4ed79589ef4a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117848) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 19 11:03:40.137139 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_db.sqlalchemy.engines [None req-133cd534-c08f-46a7-b241-c32db1c6184b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117849) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 19 11:03:40.140458 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writer thread running {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 19 11:03:40.143520 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writer thread running {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 19 11:03:40.196770 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.context [None req-dcbfff08-c38c-4dc2-aeb4-4ed79589ef4a None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),8c617d32-2606-4a09-a34c-e793c0a350a1(cell1) {{(pid=117848) load_cells /opt/stack/nova/nova/context.py:464}} Jul 19 11:03:40.199295 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.context [None req-133cd534-c08f-46a7-b241-c32db1c6184b None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),8c617d32-2606-4a09-a34c-e793c0a350a1(cell1) {{(pid=117849) load_cells /opt/stack/nova/nova/context.py:464}} Jul 19 11:03:40.200866 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dcbfff08-c38c-4dc2-aeb4-4ed79589ef4a None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:03:40.202416 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dcbfff08-c38c-4dc2-aeb4-4ed79589ef4a None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:03:40.203215 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dcbfff08-c38c-4dc2-aeb4-4ed79589ef4a None None] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:03:40.203361 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-133cd534-c08f-46a7-b241-c32db1c6184b None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:03:40.204940 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-133cd534-c08f-46a7-b241-c32db1c6184b None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:03:40.205212 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dcbfff08-c38c-4dc2-aeb4-4ed79589ef4a None None] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:03:40.205749 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-133cd534-c08f-46a7-b241-c32db1c6184b None None] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:03:40.207192 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO dbcounter [None req-dcbfff08-c38c-4dc2-aeb4-4ed79589ef4a None None] Registered counter for database nova_cell0 Jul 19 11:03:40.207729 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-133cd534-c08f-46a7-b241-c32db1c6184b None None] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:03:40.209742 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO dbcounter [None req-133cd534-c08f-46a7-b241-c32db1c6184b None None] Registered counter for database nova_cell0 Jul 19 11:03:40.211284 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO dbcounter [None req-dcbfff08-c38c-4dc2-aeb4-4ed79589ef4a None None] Registered counter for database nova_cell1 Jul 19 11:03:40.213799 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO dbcounter [None req-133cd534-c08f-46a7-b241-c32db1c6184b None None] Registered counter for database nova_cell1 Jul 19 11:03:40.218749 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_db.sqlalchemy.engines [None req-dcbfff08-c38c-4dc2-aeb4-4ed79589ef4a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117848) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 19 11:03:40.220962 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_db.sqlalchemy.engines [None req-dcbfff08-c38c-4dc2-aeb4-4ed79589ef4a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117848) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 19 11:03:40.222514 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_db.sqlalchemy.engines [None req-133cd534-c08f-46a7-b241-c32db1c6184b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117849) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 19 11:03:40.224673 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_db.sqlalchemy.engines [None req-133cd534-c08f-46a7-b241-c32db1c6184b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117849) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 19 11:03:40.225660 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writer thread running {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 19 11:03:40.232875 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writer thread running {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 19 11:03:40.233213 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writer thread running {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 19 11:03:40.364500 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writer thread running {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 19 11:03:40.375933 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO dbcounter [None req-dcbfff08-c38c-4dc2-aeb4-4ed79589ef4a None None] Registered counter for database nova_cell0 Jul 19 11:03:40.377801 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO dbcounter [None req-133cd534-c08f-46a7-b241-c32db1c6184b None None] Registered counter for database nova_cell0 Jul 19 11:03:40.383934 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_db.sqlalchemy.engines [None req-dcbfff08-c38c-4dc2-aeb4-4ed79589ef4a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117848) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 19 11:03:40.385669 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_db.sqlalchemy.engines [None req-133cd534-c08f-46a7-b241-c32db1c6184b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117849) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 19 11:03:40.388466 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writer thread running {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 19 11:03:40.390245 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writer thread running {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 19 11:03:40.705161 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: WARNING keystonemiddleware.auth_token [None req-dcbfff08-c38c-4dc2-aeb4-4ed79589ef4a None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 19 11:03:40.708122 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: WARNING keystonemiddleware.auth_token [None req-133cd534-c08f-46a7-b241-c32db1c6184b None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 19 11:03:40.713033 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: WARNING keystonemiddleware.auth_token [None req-dcbfff08-c38c-4dc2-aeb4-4ed79589ef4a None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 19 11:03:40.716877 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: WARNING keystonemiddleware.auth_token [None req-133cd534-c08f-46a7-b241-c32db1c6184b None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 19 11:03:41.055259 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: WARNING keystonemiddleware.auth_token [None req-dcbfff08-c38c-4dc2-aeb4-4ed79589ef4a None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 19 11:03:41.060069 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: WARNING keystonemiddleware.auth_token [None req-133cd534-c08f-46a7-b241-c32db1c6184b None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 19 11:03:41.061260 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: WARNING keystonemiddleware.auth_token [None req-dcbfff08-c38c-4dc2-aeb4-4ed79589ef4a None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 19 11:03:41.066329 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: WARNING keystonemiddleware.auth_token [None req-133cd534-c08f-46a7-b241-c32db1c6184b None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 19 11:03:41.067602 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x560fd77459c0 pid: 117848 (default app) Jul 19 11:03:41.073078 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x560fd77459c0 pid: 117849 (default app) Jul 19 11:04:23.116207 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_api:SELECT=3 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:04:23.118513 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=3 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:04:23.119636 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell0:SELECT=3 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:04:23.120826 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell0:SELECT=3 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:04:23.275494 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_api:SELECT=3 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:04:23.276471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=3 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:04:23.280378 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell0:SELECT=3 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:04:23.281571 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell0:SELECT=3 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:04:24.402363 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-69e2bda5-1854-499c-ac25-7391f9c29a8a tempest-VolumesGetTestJSON-1807224267 tempest-VolumesGetTestJSON-1807224267-project] Action: 'create', calling method: .version_select at 0x7f04ac326c10>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-2033905051", "metadata": {"Type": "work"}}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:04:24.553182 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-708c0ae8-9960-463c-887b-edb0969ab0f3 tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] Action: 'create', calling method: .version_select at 0x7f04ac325c10>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-1863913733"}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:04:25.932130 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-708c0ae8-9960-463c-887b-edb0969ab0f3 tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] 10.222.0.33 "POST /compute/v2.1/os-volumes" status: 200 len: 345 microversion: 2.1 time: 2.670934 Jul 19 11:04:25.941209 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 1/1] 10.222.0.33 () {60 vars in 1154 bytes} [Tue Jul 19 11:04:23 2022] POST /compute/v2.1/os-volumes => generated 345 bytes in 2675 msecs (HTTP/1.1 200) 10 headers in 448 bytes (2 switches on core 0) Jul 19 11:04:25.941209 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-9f191710-1f05-4db9-bdda-dd015238f511 tempest-VolumesTestJSON-1139155950 tempest-VolumesTestJSON-1139155950-project] Calling method '.version_select at 0x7f04ac1f0af0>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:25.996181 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-9f191710-1f05-4db9-bdda-dd015238f511 tempest-VolumesTestJSON-1139155950 tempest-VolumesTestJSON-1139155950-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/3eba237c-81de-468c-a80a-8c5e0d034324" status: 200 len: 351 microversion: 2.1 time: 0.059749 Jul 19 11:04:25.997066 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 2/2] 10.222.0.33 () {58 vars in 1244 bytes} [Tue Jul 19 11:04:25 2022] GET /compute/v2.1/os-volumes/3eba237c-81de-468c-a80a-8c5e0d034324 => generated 351 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:04:26.001981 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-5f1f56b7-e0f5-4685-b364-ab0faeb28706 tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] Calling method '.version_select at 0x7f04ac1f0d30>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:26.065137 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-69e2bda5-1854-499c-ac25-7391f9c29a8a tempest-VolumesGetTestJSON-1807224267 tempest-VolumesGetTestJSON-1807224267-project] 10.222.0.33 "POST /compute/v2.1/os-volumes" status: 200 len: 353 microversion: 2.1 time: 2.966892 Jul 19 11:04:26.066033 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 1/3] 10.222.0.33 () {60 vars in 1155 bytes} [Tue Jul 19 11:04:23 2022] POST /compute/v2.1/os-volumes => generated 353 bytes in 2969 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Jul 19 11:04:26.078766 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-41f039ae-1c74-414a-aca6-2807df980bd0 tempest-VolumesGetTestJSON-1807224267 tempest-VolumesGetTestJSON-1807224267-project] Calling method '.version_select at 0x7f04ac1f0a60>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:26.153616 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-5f1f56b7-e0f5-4685-b364-ab0faeb28706 tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/e71aa705-a579-4483-b8f5-b2f327acdbaf" status: 200 len: 345 microversion: 2.1 time: 0.155114 Jul 19 11:04:26.154183 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 3/4] 10.222.0.33 () {58 vars in 1244 bytes} [Tue Jul 19 11:04:25 2022] GET /compute/v2.1/os-volumes/e71aa705-a579-4483-b8f5-b2f327acdbaf => generated 345 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:04:26.283569 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-41f039ae-1c74-414a-aca6-2807df980bd0 tempest-VolumesGetTestJSON-1807224267 tempest-VolumesGetTestJSON-1807224267-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/2ed767a6-76c1-4f7e-b872-91b4a5583da6" status: 200 len: 353 microversion: 2.1 time: 0.209233 Jul 19 11:04:26.284669 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 2/5] 10.222.0.33 () {58 vars in 1244 bytes} [Tue Jul 19 11:04:26 2022] GET /compute/v2.1/os-volumes/2ed767a6-76c1-4f7e-b872-91b4a5583da6 => generated 353 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:04:27.168933 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-65658e82-b492-4c4e-84f9-6b6f75f429c0 tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] Calling method '.version_select at 0x7f04ac20a310>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:27.304928 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-65658e82-b492-4c4e-84f9-6b6f75f429c0 tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/e71aa705-a579-4483-b8f5-b2f327acdbaf" status: 200 len: 346 microversion: 2.1 time: 0.094946 Jul 19 11:04:27.304928 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 4/6] 10.222.0.33 () {58 vars in 1244 bytes} [Tue Jul 19 11:04:27 2022] GET /compute/v2.1/os-volumes/e71aa705-a579-4483-b8f5-b2f327acdbaf => generated 346 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:04:27.309846 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-e492c36d-fa28-4d8f-86e6-22c00ec82af4 tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] Action: 'create', calling method: .version_select at 0x7f04ac1f0d30>, body: {"snapshot": {"volume_id": "e71aa705-a579-4483-b8f5-b2f327acdbaf", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-331020655"}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:04:27.313003 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-ce981092-4cc1-48fd-8d0e-9dda0f1cefeb tempest-VolumesGetTestJSON-1807224267 tempest-VolumesGetTestJSON-1807224267-project] Calling method '.version_select at 0x7f04ac20a9d0>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:27.401015 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-ce981092-4cc1-48fd-8d0e-9dda0f1cefeb tempest-VolumesGetTestJSON-1807224267 tempest-VolumesGetTestJSON-1807224267-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/2ed767a6-76c1-4f7e-b872-91b4a5583da6" status: 200 len: 354 microversion: 2.1 time: 0.108387 Jul 19 11:04:27.401683 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 5/7] 10.222.0.33 () {58 vars in 1244 bytes} [Tue Jul 19 11:04:27 2022] GET /compute/v2.1/os-volumes/2ed767a6-76c1-4f7e-b872-91b4a5583da6 => generated 354 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:04:27.414563 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-f24209c6-4af2-4e6d-a802-401f4af171a9 tempest-VolumesGetTestJSON-1807224267 tempest-VolumesGetTestJSON-1807224267-project] Calling method '.version_select at 0x7f04ac20ac10>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:27.510046 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-f24209c6-4af2-4e6d-a802-401f4af171a9 tempest-VolumesGetTestJSON-1807224267 tempest-VolumesGetTestJSON-1807224267-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/2ed767a6-76c1-4f7e-b872-91b4a5583da6" status: 200 len: 354 microversion: 2.1 time: 0.099909 Jul 19 11:04:27.510617 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 6/8] 10.222.0.33 () {58 vars in 1244 bytes} [Tue Jul 19 11:04:27 2022] GET /compute/v2.1/os-volumes/2ed767a6-76c1-4f7e-b872-91b4a5583da6 => generated 354 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:04:27.527944 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-3a36046e-8d56-4e1f-93ea-36e6d371271e tempest-VolumesGetTestJSON-1807224267 tempest-VolumesGetTestJSON-1807224267-project] Calling method '.version_select at 0x7f04ac20a4c0>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:27.656554 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-3a36046e-8d56-4e1f-93ea-36e6d371271e tempest-VolumesGetTestJSON-1807224267 tempest-VolumesGetTestJSON-1807224267-project] 10.222.0.33 "DELETE /compute/v2.1/os-volumes/2ed767a6-76c1-4f7e-b872-91b4a5583da6" status: 202 len: 0 microversion: 2.1 time: 0.132304 Jul 19 11:04:27.662099 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 7/9] 10.222.0.33 () {58 vars in 1247 bytes} [Tue Jul 19 11:04:27 2022] DELETE /compute/v2.1/os-volumes/2ed767a6-76c1-4f7e-b872-91b4a5583da6 => generated 0 bytes in 139 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 19 11:04:27.669101 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-cc4ae611-c4fd-40a1-840b-26e0d9d6b8fc tempest-VolumesGetTestJSON-1807224267 tempest-VolumesGetTestJSON-1807224267-project] Calling method '.version_select at 0x7f04ac1f0b80>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:27.806512 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-e492c36d-fa28-4d8f-86e6-22c00ec82af4 tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] 10.222.0.33 "POST /compute/v2.1/os-snapshots" status: 200 len: 286 microversion: 2.1 time: 0.537255 Jul 19 11:04:27.807489 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 3/10] 10.222.0.33 () {60 vars in 1161 bytes} [Tue Jul 19 11:04:27 2022] POST /compute/v2.1/os-snapshots => generated 286 bytes in 540 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:04:27.826197 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-5109770a-9f7d-4fd2-ac6e-58cc01f62b19 tempest-VolumesTestJSON-1139155950 tempest-VolumesTestJSON-1139155950-project] Calling method '.version_select at 0x7f04ac203700>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:27.878698 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-cc4ae611-c4fd-40a1-840b-26e0d9d6b8fc tempest-VolumesGetTestJSON-1807224267 tempest-VolumesGetTestJSON-1807224267-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/2ed767a6-76c1-4f7e-b872-91b4a5583da6" status: 200 len: 353 microversion: 2.1 time: 0.214631 Jul 19 11:04:27.879438 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 8/11] 10.222.0.33 () {58 vars in 1244 bytes} [Tue Jul 19 11:04:27 2022] GET /compute/v2.1/os-volumes/2ed767a6-76c1-4f7e-b872-91b4a5583da6 => generated 353 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:04:27.885423 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-a13621a5-318e-4106-9e4d-e7c356ef5b08 tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] Calling method '.version_select at 0x7f04ac1f0ca0>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:27.953380 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-5109770a-9f7d-4fd2-ac6e-58cc01f62b19 tempest-VolumesTestJSON-1139155950 tempest-VolumesTestJSON-1139155950-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/c7ab5c71-2862-4848-8299-b98bd8127bf1" status: 200 len: 351 microversion: 2.1 time: 0.137844 Jul 19 11:04:27.954059 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 4/12] 10.222.0.33 () {58 vars in 1244 bytes} [Tue Jul 19 11:04:27 2022] GET /compute/v2.1/os-volumes/c7ab5c71-2862-4848-8299-b98bd8127bf1 => generated 351 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:04:27.970498 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-a13621a5-318e-4106-9e4d-e7c356ef5b08 tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] 10.222.0.33 "GET /compute/v2.1/os-snapshots/df69d2d5-460c-4f5c-8641-c9c3b480767c" status: 200 len: 286 microversion: 2.1 time: 0.089794 Jul 19 11:04:27.971648 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 9/13] 10.222.0.33 () {58 vars in 1250 bytes} [Tue Jul 19 11:04:27 2022] GET /compute/v2.1/os-snapshots/df69d2d5-460c-4f5c-8641-c9c3b480767c => generated 286 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:04:27.982440 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-31432919-17bd-453d-bbe6-bbb41f3d31e4 tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] Calling method '.version_select at 0x7f04ac203dc0>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:28.061596 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-31432919-17bd-453d-bbe6-bbb41f3d31e4 tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] 10.222.0.33 "GET /compute/v2.1/os-snapshots" status: 200 len: 289 microversion: 2.1 time: 0.083306 Jul 19 11:04:28.062527 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 5/14] 10.222.0.33 () {58 vars in 1139 bytes} [Tue Jul 19 11:04:27 2022] GET /compute/v2.1/os-snapshots => generated 289 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:04:28.074375 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-4e42b099-9dc7-4698-900a-059b6783ca34 tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] Calling method '.version_select at 0x7f04ac20a3a0>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:28.142076 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-4e42b099-9dc7-4698-900a-059b6783ca34 tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] 10.222.0.33 "GET /compute/v2.1/os-snapshots/df69d2d5-460c-4f5c-8641-c9c3b480767c" status: 200 len: 286 microversion: 2.1 time: 0.073102 Jul 19 11:04:28.143164 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 10/15] 10.222.0.33 () {58 vars in 1250 bytes} [Tue Jul 19 11:04:28 2022] GET /compute/v2.1/os-snapshots/df69d2d5-460c-4f5c-8641-c9c3b480767c => generated 286 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:04:28.891670 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-d7b7437a-a34a-41a6-b721-beef8f180fed tempest-VolumesGetTestJSON-1807224267 tempest-VolumesGetTestJSON-1807224267-project] Calling method '.version_select at 0x7f04ac2033a0>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:28.958025 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.wsgi [None req-d7b7437a-a34a-41a6-b721-beef8f180fed tempest-VolumesGetTestJSON-1807224267 tempest-VolumesGetTestJSON-1807224267-project] HTTP exception thrown: Volume 2ed767a6-76c1-4f7e-b872-91b4a5583da6 could not be found. Jul 19 11:04:28.966987 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-d7b7437a-a34a-41a6-b721-beef8f180fed tempest-VolumesGetTestJSON-1807224267 tempest-VolumesGetTestJSON-1807224267-project] Request method failure captured: Jul 19 11:04:28.966987 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: request: GET /compute/v2.1/os-volumes/2ed767a6-76c1-4f7e-b872-91b4a5583da6 HTTP/1.1 Jul 19 11:04:28.966987 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept: application/json Jul 19 11:04:28.966987 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept-Encoding: identity Jul 19 11:04:28.966987 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Connection: close Jul 19 11:04:28.966987 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Length: 0 Jul 19 11:04:28.966987 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Type: application/json Jul 19 11:04:28.966987 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Host: 10.222.0.33 Jul 19 11:04:28.966987 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Openstack-System-Scope: None Jul 19 11:04:28.966987 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: User-Agent: python-urllib3/1.26.9 Jul 19 11:04:28.966987 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Auth-Token: gAAAAABi1o-2vCgP4d_D3A0xWwYA1mf-4XCSTIPq7Q1yM97OVN8E1r_Ct7e4Z9l735Oqw5XsUd-tgyDa3e2CXaGAnax6QMduvYamY-soMcz67-agrkGM_7rfDJGl-K7huwDhRjr5RB_tobElxuhfCj1O6eKxGCJ6SgrVVve52ooPcfq0UlVKbOg Jul 19 11:04:28.966987 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Id: None Jul 19 11:04:28.966987 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Name: None Jul 19 11:04:28.966987 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Identity-Status: Confirmed Jul 19 11:04:28.966987 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Is-Admin-Project: True Jul 19 11:04:28.966987 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Id: default Jul 19 11:04:28.966987 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Name: Default Jul 19 11:04:28.966987 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Id: 7d0d678908014f8dae55bc6475b84680 Jul 19 11:04:28.966987 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Name: tempest-VolumesGetTestJSON-1807224267 Jul 19 11:04:28.966987 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Role: member,reader Jul 19 11:04:28.966987 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Roles: member,reader Jul 19 11:04:28.968584 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/7d0d678908014f8dae55bc6475b84680"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/7d0d678908014f8dae55bc6475b84680"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/7d0d678908014f8dae55bc6475b84680"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_7d0d678908014f8dae55bc6475b84680"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:04:28.968584 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant: tempest-VolumesGetTestJSON-1807224267 Jul 19 11:04:28.968584 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Id: 7d0d678908014f8dae55bc6475b84680 Jul 19 11:04:28.968584 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Name: tempest-VolumesGetTestJSON-1807224267 Jul 19 11:04:28.968584 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User: tempest-VolumesGetTestJSON-1807224267-project Jul 19 11:04:28.968584 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Id: default Jul 19 11:04:28.968584 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Name: Default Jul 19 11:04:28.968584 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Id: e68eb1137cc845148ba17a3d5c46f878 Jul 19 11:04:28.968584 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Name: tempest-VolumesGetTestJSON-1807224267-project Jul 19 11:04:28.968584 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: method: Jul 19 11:04:28.968584 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: exception: Volume 2ed767a6-76c1-4f7e-b872-91b4a5583da6 could not be found. Jul 19 11:04:28.968584 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: action_args: {'id': '2ed767a6-76c1-4f7e-b872-91b4a5583da6'} Jul 19 11:04:28.968584 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:04:28.969610 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:04:28.969610 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 19 11:04:28.969610 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 19 11:04:28.969610 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 19 11:04:28.969610 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 19 11:04:28.969610 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 19 11:04:28.969610 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 19 11:04:28.969610 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 19 11:04:28.969610 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 19 11:04:28.969610 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 19 11:04:28.969610 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 19 11:04:28.969610 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 19 11:04:28.969610 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 19 11:04:28.969610 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 19 11:04:28.969610 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 19 11:04:28.969610 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 2ed767a6-76c1-4f7e-b872-91b4a5583da6 could not be found. (HTTP 404) (Request-ID: req-debb1426-7186-45ba-83bb-7793174938fa) Jul 19 11:04:28.969610 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:04:28.969610 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:04:28.969610 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:04:28.970953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:04:28.970953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Jul 19 11:04:28.970953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Jul 19 11:04:28.970953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 19 11:04:28.970953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 19 11:04:28.970953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 19 11:04:28.970953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 19 11:04:28.970953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 19 11:04:28.970953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 19 11:04:28.970953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 19 11:04:28.970953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 19 11:04:28.970953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 19 11:04:28.970953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 19 11:04:28.970953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 19 11:04:28.970953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 19 11:04:28.970953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 19 11:04:28.970953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 19 11:04:28.970953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 19 11:04:28.970953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 19 11:04:28.972308 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 19 11:04:28.972308 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 19 11:04:28.972308 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 19 11:04:28.972308 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 19 11:04:28.972308 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 2ed767a6-76c1-4f7e-b872-91b4a5583da6 could not be found. Jul 19 11:04:28.972308 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:04:28.972308 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:04:28.972308 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:04:28.972308 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:04:28.972308 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:04:28.972308 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:04:28.972308 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:04:28.972308 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:04:28.972308 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 19 11:04:28.972308 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 19 11:04:28.972308 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:04:28.972308 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:04:28.972308 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Jul 19 11:04:28.972308 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:04:28.973825 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 2ed767a6-76c1-4f7e-b872-91b4a5583da6 could not be found. Jul 19 11:04:28.973825 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:04:28.973825 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:04:28.973825 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:04:28.973825 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:04:28.973825 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:04:28.973825 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:04:28.973825 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:04:28.973825 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:04:28.973825 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 2ed767a6-76c1-4f7e-b872-91b4a5583da6 could not be found. Jul 19 11:04:28.973825 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:04:28.986027 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-d7b7437a-a34a-41a6-b721-beef8f180fed tempest-VolumesGetTestJSON-1807224267 tempest-VolumesGetTestJSON-1807224267-project] Returning 404 to user: Volume 2ed767a6-76c1-4f7e-b872-91b4a5583da6 could not be found. {{(pid=117848) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:04:28.987827 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-d7b7437a-a34a-41a6-b721-beef8f180fed tempest-VolumesGetTestJSON-1807224267 tempest-VolumesGetTestJSON-1807224267-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/2ed767a6-76c1-4f7e-b872-91b4a5583da6" status: 404 len: 109 microversion: 2.1 time: 0.100132 Jul 19 11:04:28.989213 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 6/16] 10.222.0.33 () {58 vars in 1244 bytes} [Tue Jul 19 11:04:28 2022] GET /compute/v2.1/os-volumes/2ed767a6-76c1-4f7e-b872-91b4a5583da6 => generated 109 bytes in 102 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:04:29.167728 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-ae4d554c-d83e-4355-acff-ac74c85a0bce tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] Calling method '.version_select at 0x7f04ac20ab80>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:29.211445 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-ae4d554c-d83e-4355-acff-ac74c85a0bce tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] 10.222.0.33 "GET /compute/v2.1/os-snapshots/df69d2d5-460c-4f5c-8641-c9c3b480767c" status: 200 len: 286 microversion: 2.1 time: 0.060605 Jul 19 11:04:29.214647 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 11/17] 10.222.0.33 () {58 vars in 1250 bytes} [Tue Jul 19 11:04:29 2022] GET /compute/v2.1/os-snapshots/df69d2d5-460c-4f5c-8641-c9c3b480767c => generated 286 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:04:30.225139 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-779bf83a-a667-48ef-8e16-39cafd768117 tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] Calling method '.version_select at 0x7f04acb87040>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:30.262107 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-779bf83a-a667-48ef-8e16-39cafd768117 tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] 10.222.0.33 "GET /compute/v2.1/os-snapshots/df69d2d5-460c-4f5c-8641-c9c3b480767c" status: 200 len: 286 microversion: 2.1 time: 0.042366 Jul 19 11:04:30.266910 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 7/18] 10.222.0.33 () {58 vars in 1250 bytes} [Tue Jul 19 11:04:30 2022] GET /compute/v2.1/os-snapshots/df69d2d5-460c-4f5c-8641-c9c3b480767c => generated 286 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:04:31.277275 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-d64b4e51-651e-46df-9472-d8efe97946e9 tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] Calling method '.version_select at 0x7f04ac214310>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:31.354619 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-d64b4e51-651e-46df-9472-d8efe97946e9 tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] 10.222.0.33 "GET /compute/v2.1/os-snapshots/df69d2d5-460c-4f5c-8641-c9c3b480767c" status: 200 len: 287 microversion: 2.1 time: 0.079342 Jul 19 11:04:31.356639 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 12/19] 10.222.0.33 () {58 vars in 1250 bytes} [Tue Jul 19 11:04:31 2022] GET /compute/v2.1/os-snapshots/df69d2d5-460c-4f5c-8641-c9c3b480767c => generated 287 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:04:31.373373 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-1292b232-e03e-4a19-abd4-9979b59b8bc8 tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] Calling method '.version_select at 0x7f04ac1604c0>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:31.530035 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-1292b232-e03e-4a19-abd4-9979b59b8bc8 tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] 10.222.0.33 "DELETE /compute/v2.1/os-snapshots/df69d2d5-460c-4f5c-8641-c9c3b480767c" status: 202 len: 0 microversion: 2.1 time: 0.167729 Jul 19 11:04:31.530891 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 8/20] 10.222.0.33 () {58 vars in 1253 bytes} [Tue Jul 19 11:04:31 2022] DELETE /compute/v2.1/os-snapshots/df69d2d5-460c-4f5c-8641-c9c3b480767c => generated 0 bytes in 169 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 19 11:04:31.543053 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-f335d953-bc3f-4e7d-b570-c0c1b144b88d tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] Calling method '.version_select at 0x7f04ac20ab80>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:31.628902 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-f335d953-bc3f-4e7d-b570-c0c1b144b88d tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] 10.222.0.33 "GET /compute/v2.1/os-snapshots/df69d2d5-460c-4f5c-8641-c9c3b480767c" status: 200 len: 286 microversion: 2.1 time: 0.085505 Jul 19 11:04:31.629358 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 13/21] 10.222.0.33 () {58 vars in 1250 bytes} [Tue Jul 19 11:04:31 2022] GET /compute/v2.1/os-snapshots/df69d2d5-460c-4f5c-8641-c9c3b480767c => generated 286 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:04:31.762939 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-88cf575c-6a6e-4970-90b1-74e8af7b4c86 tempest-VolumesTestJSON-1139155950 tempest-VolumesTestJSON-1139155950-project] Calling method '.version_select at 0x7f04ac160b80>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:31.888640 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-88cf575c-6a6e-4970-90b1-74e8af7b4c86 tempest-VolumesTestJSON-1139155950 tempest-VolumesTestJSON-1139155950-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/7e448537-f0a0-4dae-84af-6f1d3e9d1372" status: 200 len: 350 microversion: 2.1 time: 0.129371 Jul 19 11:04:31.889487 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 9/22] 10.222.0.33 () {58 vars in 1244 bytes} [Tue Jul 19 11:04:31 2022] GET /compute/v2.1/os-volumes/7e448537-f0a0-4dae-84af-6f1d3e9d1372 => generated 350 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:04:31.901721 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-bdc1fc88-69c8-4db1-a2dc-a122d23e1649 tempest-VolumesTestJSON-1139155950 tempest-VolumesTestJSON-1139155950-project] Calling method '.version_select at 0x7f04ac20a4c0>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:32.090054 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-bdc1fc88-69c8-4db1-a2dc-a122d23e1649 tempest-VolumesTestJSON-1139155950 tempest-VolumesTestJSON-1139155950-project] 10.222.0.33 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.192518 Jul 19 11:04:32.090902 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 14/23] 10.222.0.33 () {58 vars in 1133 bytes} [Tue Jul 19 11:04:31 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:04:32.104116 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-638b6b19-b2d4-484b-bf50-963ba81bb862 tempest-VolumesTestJSON-1139155950 tempest-VolumesTestJSON-1139155950-project] Calling method '.version_select at 0x7f04ac1780d0>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:32.361508 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-638b6b19-b2d4-484b-bf50-963ba81bb862 tempest-VolumesTestJSON-1139155950 tempest-VolumesTestJSON-1139155950-project] 10.222.0.33 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.261590 Jul 19 11:04:32.362491 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 10/24] 10.222.0.33 () {58 vars in 1148 bytes} [Tue Jul 19 11:04:32 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 694 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:04:32.375010 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-7fcd1bd4-814f-4534-8488-b27d24e8470c tempest-VolumesTestJSON-1139155950 tempest-VolumesTestJSON-1139155950-project] Calling method '.version_select at 0x7f04ac214430>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:32.552183 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-7fcd1bd4-814f-4534-8488-b27d24e8470c tempest-VolumesTestJSON-1139155950 tempest-VolumesTestJSON-1139155950-project] 10.222.0.33 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.181038 Jul 19 11:04:32.553114 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 15/25] 10.222.0.33 () {58 vars in 1133 bytes} [Tue Jul 19 11:04:32 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:04:32.565290 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-3f12700f-879b-4660-a116-1de67a624f2a tempest-VolumesTestJSON-1139155950 tempest-VolumesTestJSON-1139155950-project] Calling method '.version_select at 0x7f04ac160940>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:32.640847 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-a702ba02-0fe0-4f79-9871-4daa50e0c6fe tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] Calling method '.version_select at 0x7f04ac20a1f0>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:32.695184 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.wsgi [None req-a702ba02-0fe0-4f79-9871-4daa50e0c6fe tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] HTTP exception thrown: Snapshot df69d2d5-460c-4f5c-8641-c9c3b480767c could not be found. Jul 19 11:04:32.699564 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-a702ba02-0fe0-4f79-9871-4daa50e0c6fe tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] Request method failure captured: Jul 19 11:04:32.699564 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: request: GET /compute/v2.1/os-snapshots/df69d2d5-460c-4f5c-8641-c9c3b480767c HTTP/1.1 Jul 19 11:04:32.699564 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept: application/json Jul 19 11:04:32.699564 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept-Encoding: identity Jul 19 11:04:32.699564 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Connection: close Jul 19 11:04:32.699564 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Length: 0 Jul 19 11:04:32.699564 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Type: application/json Jul 19 11:04:32.699564 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Host: 10.222.0.33 Jul 19 11:04:32.699564 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Openstack-System-Scope: None Jul 19 11:04:32.699564 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: User-Agent: python-urllib3/1.26.9 Jul 19 11:04:32.699564 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Auth-Token: gAAAAABi1o-3Acst1r879RI9dFHt7UJWzPFghEd6G_jFMFMtxp3NKS13B-_BnHBn63S7pGqqoqx8J4I5JTifCCqB4Ih1J-wEPKLiQXmZOvOEqSvzdQOCkUbibasYuhFeMPZx3Vbxy2zpZM0DXFRSV8VGjxwthpDP54keDwSCrvQRs2RPv2cJRzA Jul 19 11:04:32.699564 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Id: None Jul 19 11:04:32.699564 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Name: None Jul 19 11:04:32.699564 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Identity-Status: Confirmed Jul 19 11:04:32.699564 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Is-Admin-Project: True Jul 19 11:04:32.699564 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Id: default Jul 19 11:04:32.699564 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Name: Default Jul 19 11:04:32.699564 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Id: 888be6c04e3e4248a000a1c601a1c8ed Jul 19 11:04:32.699564 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1903635009 Jul 19 11:04:32.699564 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Role: member,reader Jul 19 11:04:32.699564 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Roles: member,reader Jul 19 11:04:32.700631 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/888be6c04e3e4248a000a1c601a1c8ed"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/888be6c04e3e4248a000a1c601a1c8ed"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/888be6c04e3e4248a000a1c601a1c8ed"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_888be6c04e3e4248a000a1c601a1c8ed"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:04:32.700631 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1903635009 Jul 19 11:04:32.700631 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Id: 888be6c04e3e4248a000a1c601a1c8ed Jul 19 11:04:32.700631 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1903635009 Jul 19 11:04:32.700631 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User: tempest-VolumesSnapshotsTestJSON-1903635009-project Jul 19 11:04:32.700631 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Id: default Jul 19 11:04:32.700631 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Name: Default Jul 19 11:04:32.700631 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Id: 2c9c1fff948941b8baacac536fc09724 Jul 19 11:04:32.700631 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1903635009-project Jul 19 11:04:32.700631 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: method: Jul 19 11:04:32.700631 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: exception: Snapshot df69d2d5-460c-4f5c-8641-c9c3b480767c could not be found. Jul 19 11:04:32.700631 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: action_args: {'id': 'df69d2d5-460c-4f5c-8641-c9c3b480767c'} Jul 19 11:04:32.700631 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:04:32.701562 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:04:32.701562 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Jul 19 11:04:32.701562 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Jul 19 11:04:32.701562 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Jul 19 11:04:32.701562 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Jul 19 11:04:32.701562 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Jul 19 11:04:32.701562 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Jul 19 11:04:32.701562 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 19 11:04:32.701562 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 19 11:04:32.701562 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 19 11:04:32.701562 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 19 11:04:32.701562 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 19 11:04:32.701562 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 19 11:04:32.701562 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 19 11:04:32.701562 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 19 11:04:32.701562 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot df69d2d5-460c-4f5c-8641-c9c3b480767c could not be found. (HTTP 404) (Request-ID: req-c3656c94-f7e2-4709-8bb7-e14701b47d64) Jul 19 11:04:32.701562 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:04:32.701562 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:04:32.702371 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:04:32.702371 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:04:32.702371 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Jul 19 11:04:32.702371 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Jul 19 11:04:32.702371 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 19 11:04:32.702371 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 19 11:04:32.702371 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Jul 19 11:04:32.702371 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Jul 19 11:04:32.702371 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 19 11:04:32.702371 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 19 11:04:32.702371 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Jul 19 11:04:32.702371 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Jul 19 11:04:32.702371 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Jul 19 11:04:32.702371 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Jul 19 11:04:32.702371 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Jul 19 11:04:32.702371 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Jul 19 11:04:32.702371 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 19 11:04:32.702371 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 19 11:04:32.702371 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 19 11:04:32.703805 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 19 11:04:32.703805 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 19 11:04:32.703805 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 19 11:04:32.703805 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 19 11:04:32.703805 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 19 11:04:32.703805 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot df69d2d5-460c-4f5c-8641-c9c3b480767c could not be found. Jul 19 11:04:32.703805 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:04:32.703805 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:04:32.703805 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:04:32.703805 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:04:32.703805 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:04:32.703805 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:04:32.703805 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:04:32.703805 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:04:32.703805 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 19 11:04:32.703805 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 19 11:04:32.703805 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:04:32.703805 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:04:32.703805 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Jul 19 11:04:32.703805 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:04:32.705684 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot df69d2d5-460c-4f5c-8641-c9c3b480767c could not be found. Jul 19 11:04:32.705684 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:04:32.705684 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:04:32.705684 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:04:32.705684 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:04:32.705684 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:04:32.705684 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:04:32.705684 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:04:32.705684 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:04:32.705684 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot df69d2d5-460c-4f5c-8641-c9c3b480767c could not be found. Jul 19 11:04:32.705684 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:04:32.708221 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-a702ba02-0fe0-4f79-9871-4daa50e0c6fe tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] Returning 404 to user: Snapshot df69d2d5-460c-4f5c-8641-c9c3b480767c could not be found. {{(pid=117849) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:04:32.708771 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-a702ba02-0fe0-4f79-9871-4daa50e0c6fe tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] 10.222.0.33 "GET /compute/v2.1/os-snapshots/df69d2d5-460c-4f5c-8641-c9c3b480767c" status: 404 len: 111 microversion: 2.1 time: 0.071730 Jul 19 11:04:32.710129 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 16/26] 10.222.0.33 () {58 vars in 1250 bytes} [Tue Jul 19 11:04:32 2022] GET /compute/v2.1/os-snapshots/df69d2d5-460c-4f5c-8641-c9c3b480767c => generated 111 bytes in 74 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:04:32.734678 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-82f5232a-7882-40bd-a896-618fb8be0124 tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] Calling method '.version_select at 0x7f04acb86040>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:32.800625 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-3f12700f-879b-4660-a116-1de67a624f2a tempest-VolumesTestJSON-1139155950 tempest-VolumesTestJSON-1139155950-project] 10.222.0.33 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.239028 Jul 19 11:04:32.801662 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 11/27] 10.222.0.33 () {58 vars in 1166 bytes} [Tue Jul 19 11:04:32 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 354 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:04:32.813945 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-7048557d-ac40-4558-a542-8d435562935e tempest-VolumesTestJSON-1139155950 tempest-VolumesTestJSON-1139155950-project] Calling method '.version_select at 0x7f04ac203940>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:32.884620 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-82f5232a-7882-40bd-a896-618fb8be0124 tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] 10.222.0.33 "DELETE /compute/v2.1/os-volumes/e71aa705-a579-4483-b8f5-b2f327acdbaf" status: 202 len: 0 microversion: 2.1 time: 0.169119 Jul 19 11:04:32.885342 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 17/28] 10.222.0.33 () {58 vars in 1247 bytes} [Tue Jul 19 11:04:32 2022] DELETE /compute/v2.1/os-volumes/e71aa705-a579-4483-b8f5-b2f327acdbaf => generated 0 bytes in 171 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 19 11:04:32.893987 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-f85fe1ef-7327-4a38-b10d-1dc5d7e8b509 tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] Calling method '.version_select at 0x7f04ac214790>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:33.004904 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-f85fe1ef-7327-4a38-b10d-1dc5d7e8b509 tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/e71aa705-a579-4483-b8f5-b2f327acdbaf" status: 200 len: 345 microversion: 2.1 time: 0.115051 Jul 19 11:04:33.011330 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 18/29] 10.222.0.33 () {58 vars in 1244 bytes} [Tue Jul 19 11:04:32 2022] GET /compute/v2.1/os-volumes/e71aa705-a579-4483-b8f5-b2f327acdbaf => generated 345 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:04:33.146405 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-7048557d-ac40-4558-a542-8d435562935e tempest-VolumesTestJSON-1139155950 tempest-VolumesTestJSON-1139155950-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.335790 Jul 19 11:04:33.147903 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 12/30] 10.222.0.33 () {58 vars in 1169 bytes} [Tue Jul 19 11:04:32 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 694 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:04:33.161421 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-abfa19e2-f3a4-4ff2-909c-0db6411e1e16 tempest-VolumesTestJSON-1139155950 tempest-VolumesTestJSON-1139155950-project] Calling method '.version_select at 0x7f04ac214b80>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:33.422477 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-abfa19e2-f3a4-4ff2-909c-0db6411e1e16 tempest-VolumesTestJSON-1139155950 tempest-VolumesTestJSON-1139155950-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.265603 Jul 19 11:04:33.423692 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 19/31] 10.222.0.33 () {58 vars in 1154 bytes} [Tue Jul 19 11:04:33 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:04:33.435954 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-f78cb7a3-14bc-48c7-9e49-3c6106450667 tempest-VolumesTestJSON-1139155950 tempest-VolumesTestJSON-1139155950-project] Calling method '.version_select at 0x7f04ac178550>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:33.730008 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-f78cb7a3-14bc-48c7-9e49-3c6106450667 tempest-VolumesTestJSON-1139155950 tempest-VolumesTestJSON-1139155950-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.297765 Jul 19 11:04:33.730569 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 13/32] 10.222.0.33 () {58 vars in 1187 bytes} [Tue Jul 19 11:04:33 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 354 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:04:33.743069 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-58462f52-805f-4c70-95f7-ca3515582b64 tempest-VolumesTestJSON-1139155950 tempest-VolumesTestJSON-1139155950-project] Calling method '.version_select at 0x7f04ac219820>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:33.978793 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-58462f52-805f-4c70-95f7-ca3515582b64 tempest-VolumesTestJSON-1139155950 tempest-VolumesTestJSON-1139155950-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.240137 Jul 19 11:04:33.979371 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 20/33] 10.222.0.33 () {58 vars in 1154 bytes} [Tue Jul 19 11:04:33 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:04:34.017708 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-619c2dab-9094-414d-a333-01ffad340e4b tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] Calling method '.version_select at 0x7f04ac219790>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:34.086040 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.wsgi [None req-619c2dab-9094-414d-a333-01ffad340e4b tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] HTTP exception thrown: Volume e71aa705-a579-4483-b8f5-b2f327acdbaf could not be found. Jul 19 11:04:34.088302 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-619c2dab-9094-414d-a333-01ffad340e4b tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] Request method failure captured: Jul 19 11:04:34.088302 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: request: GET /compute/v2.1/os-volumes/e71aa705-a579-4483-b8f5-b2f327acdbaf HTTP/1.1 Jul 19 11:04:34.088302 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept: application/json Jul 19 11:04:34.088302 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept-Encoding: identity Jul 19 11:04:34.088302 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Connection: close Jul 19 11:04:34.088302 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Length: 0 Jul 19 11:04:34.088302 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Type: application/json Jul 19 11:04:34.088302 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Host: 10.222.0.33 Jul 19 11:04:34.088302 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Openstack-System-Scope: None Jul 19 11:04:34.088302 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: User-Agent: python-urllib3/1.26.9 Jul 19 11:04:34.088302 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Auth-Token: gAAAAABi1o-3Acst1r879RI9dFHt7UJWzPFghEd6G_jFMFMtxp3NKS13B-_BnHBn63S7pGqqoqx8J4I5JTifCCqB4Ih1J-wEPKLiQXmZOvOEqSvzdQOCkUbibasYuhFeMPZx3Vbxy2zpZM0DXFRSV8VGjxwthpDP54keDwSCrvQRs2RPv2cJRzA Jul 19 11:04:34.088302 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Id: None Jul 19 11:04:34.088302 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Name: None Jul 19 11:04:34.088302 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Identity-Status: Confirmed Jul 19 11:04:34.088302 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Is-Admin-Project: True Jul 19 11:04:34.088302 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Id: default Jul 19 11:04:34.088302 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Name: Default Jul 19 11:04:34.088302 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Id: 888be6c04e3e4248a000a1c601a1c8ed Jul 19 11:04:34.088302 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1903635009 Jul 19 11:04:34.088302 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Role: member,reader Jul 19 11:04:34.088302 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Roles: member,reader Jul 19 11:04:34.089323 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/888be6c04e3e4248a000a1c601a1c8ed"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/888be6c04e3e4248a000a1c601a1c8ed"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/888be6c04e3e4248a000a1c601a1c8ed"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_888be6c04e3e4248a000a1c601a1c8ed"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:04:34.089323 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1903635009 Jul 19 11:04:34.089323 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Id: 888be6c04e3e4248a000a1c601a1c8ed Jul 19 11:04:34.089323 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1903635009 Jul 19 11:04:34.089323 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User: tempest-VolumesSnapshotsTestJSON-1903635009-project Jul 19 11:04:34.089323 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Id: default Jul 19 11:04:34.089323 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Name: Default Jul 19 11:04:34.089323 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Id: 2c9c1fff948941b8baacac536fc09724 Jul 19 11:04:34.089323 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1903635009-project Jul 19 11:04:34.089323 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: method: Jul 19 11:04:34.089323 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: exception: Volume e71aa705-a579-4483-b8f5-b2f327acdbaf could not be found. Jul 19 11:04:34.089323 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: action_args: {'id': 'e71aa705-a579-4483-b8f5-b2f327acdbaf'} Jul 19 11:04:34.089323 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:04:34.090014 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:04:34.090014 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 19 11:04:34.090014 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 19 11:04:34.090014 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 19 11:04:34.090014 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 19 11:04:34.090014 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 19 11:04:34.090014 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 19 11:04:34.090014 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 19 11:04:34.090014 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 19 11:04:34.090014 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 19 11:04:34.090014 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 19 11:04:34.090014 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 19 11:04:34.090014 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 19 11:04:34.090014 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 19 11:04:34.090014 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 19 11:04:34.090014 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume e71aa705-a579-4483-b8f5-b2f327acdbaf could not be found. (HTTP 404) (Request-ID: req-cb231aae-7839-47f4-b264-94285965d62b) Jul 19 11:04:34.090014 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:04:34.090014 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:04:34.090014 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:04:34.090901 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:04:34.090901 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Jul 19 11:04:34.090901 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Jul 19 11:04:34.090901 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 19 11:04:34.090901 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 19 11:04:34.090901 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 19 11:04:34.090901 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 19 11:04:34.090901 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 19 11:04:34.090901 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 19 11:04:34.090901 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 19 11:04:34.090901 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 19 11:04:34.090901 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 19 11:04:34.090901 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 19 11:04:34.090901 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 19 11:04:34.090901 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 19 11:04:34.090901 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 19 11:04:34.090901 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 19 11:04:34.090901 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 19 11:04:34.090901 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 19 11:04:34.091891 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 19 11:04:34.091891 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 19 11:04:34.091891 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 19 11:04:34.091891 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 19 11:04:34.091891 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume e71aa705-a579-4483-b8f5-b2f327acdbaf could not be found. Jul 19 11:04:34.091891 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:04:34.091891 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:04:34.091891 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:04:34.091891 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:04:34.091891 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:04:34.091891 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:04:34.091891 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:04:34.091891 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:04:34.091891 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 19 11:04:34.091891 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 19 11:04:34.091891 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:04:34.091891 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:04:34.091891 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Jul 19 11:04:34.091891 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:04:34.093966 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume e71aa705-a579-4483-b8f5-b2f327acdbaf could not be found. Jul 19 11:04:34.093966 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:04:34.093966 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:04:34.093966 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:04:34.093966 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:04:34.093966 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:04:34.093966 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:04:34.093966 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:04:34.093966 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:04:34.093966 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume e71aa705-a579-4483-b8f5-b2f327acdbaf could not be found. Jul 19 11:04:34.093966 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:04:34.093966 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-619c2dab-9094-414d-a333-01ffad340e4b tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] Returning 404 to user: Volume e71aa705-a579-4483-b8f5-b2f327acdbaf could not be found. {{(pid=117849) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:04:34.093966 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-619c2dab-9094-414d-a333-01ffad340e4b tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/e71aa705-a579-4483-b8f5-b2f327acdbaf" status: 404 len: 109 microversion: 2.1 time: 0.075762 Jul 19 11:04:34.093966 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 21/34] 10.222.0.33 () {58 vars in 1244 bytes} [Tue Jul 19 11:04:34 2022] GET /compute/v2.1/os-volumes/e71aa705-a579-4483-b8f5-b2f327acdbaf => generated 109 bytes in 77 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 19 11:04:34.106362 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-97ff25d0-9832-4896-94dc-3aa6c86896f6 tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] Calling method '.version_select at 0x7f04ac2145e0>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:34.208956 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.wsgi [None req-97ff25d0-9832-4896-94dc-3aa6c86896f6 tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] HTTP exception thrown: Volume e71aa705-a579-4483-b8f5-b2f327acdbaf could not be found. Jul 19 11:04:34.211635 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-97ff25d0-9832-4896-94dc-3aa6c86896f6 tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] Request method failure captured: Jul 19 11:04:34.211635 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: request: DELETE /compute/v2.1/os-volumes/e71aa705-a579-4483-b8f5-b2f327acdbaf HTTP/1.1 Jul 19 11:04:34.211635 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept: application/json Jul 19 11:04:34.211635 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept-Encoding: identity Jul 19 11:04:34.211635 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Connection: close Jul 19 11:04:34.211635 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Length: 0 Jul 19 11:04:34.211635 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Type: application/json Jul 19 11:04:34.211635 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Host: 10.222.0.33 Jul 19 11:04:34.211635 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Openstack-System-Scope: None Jul 19 11:04:34.211635 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: User-Agent: python-urllib3/1.26.9 Jul 19 11:04:34.211635 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Auth-Token: gAAAAABi1o-3Acst1r879RI9dFHt7UJWzPFghEd6G_jFMFMtxp3NKS13B-_BnHBn63S7pGqqoqx8J4I5JTifCCqB4Ih1J-wEPKLiQXmZOvOEqSvzdQOCkUbibasYuhFeMPZx3Vbxy2zpZM0DXFRSV8VGjxwthpDP54keDwSCrvQRs2RPv2cJRzA Jul 19 11:04:34.211635 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Id: None Jul 19 11:04:34.211635 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Name: None Jul 19 11:04:34.211635 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Identity-Status: Confirmed Jul 19 11:04:34.211635 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Is-Admin-Project: True Jul 19 11:04:34.211635 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Id: default Jul 19 11:04:34.211635 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Name: Default Jul 19 11:04:34.211635 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Id: 888be6c04e3e4248a000a1c601a1c8ed Jul 19 11:04:34.211635 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1903635009 Jul 19 11:04:34.211635 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Role: member,reader Jul 19 11:04:34.211635 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Roles: member,reader Jul 19 11:04:34.212912 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/888be6c04e3e4248a000a1c601a1c8ed"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/888be6c04e3e4248a000a1c601a1c8ed"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/888be6c04e3e4248a000a1c601a1c8ed"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_888be6c04e3e4248a000a1c601a1c8ed"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:04:34.212912 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1903635009 Jul 19 11:04:34.212912 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Id: 888be6c04e3e4248a000a1c601a1c8ed Jul 19 11:04:34.212912 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1903635009 Jul 19 11:04:34.212912 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User: tempest-VolumesSnapshotsTestJSON-1903635009-project Jul 19 11:04:34.212912 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Id: default Jul 19 11:04:34.212912 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Name: Default Jul 19 11:04:34.212912 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Id: 2c9c1fff948941b8baacac536fc09724 Jul 19 11:04:34.212912 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1903635009-project Jul 19 11:04:34.212912 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: method: Jul 19 11:04:34.212912 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: exception: Volume e71aa705-a579-4483-b8f5-b2f327acdbaf could not be found. Jul 19 11:04:34.212912 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: action_args: {'id': 'e71aa705-a579-4483-b8f5-b2f327acdbaf'} Jul 19 11:04:34.212912 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:04:34.213542 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:04:34.213542 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 19 11:04:34.213542 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 19 11:04:34.213542 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 19 11:04:34.213542 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 19 11:04:34.213542 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 19 11:04:34.213542 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 19 11:04:34.213542 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 19 11:04:34.213542 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 19 11:04:34.213542 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 19 11:04:34.213542 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 19 11:04:34.213542 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 19 11:04:34.213542 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 19 11:04:34.213542 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 19 11:04:34.213542 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 19 11:04:34.213542 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume e71aa705-a579-4483-b8f5-b2f327acdbaf could not be found. (HTTP 404) (Request-ID: req-894e1bfd-b791-4ac5-94e3-9bb0bda1fc6a) Jul 19 11:04:34.213542 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:04:34.213542 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:04:34.213542 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:04:34.214399 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:04:34.214399 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Jul 19 11:04:34.214399 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Jul 19 11:04:34.214399 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 19 11:04:34.214399 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 19 11:04:34.214399 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 19 11:04:34.214399 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 19 11:04:34.214399 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 19 11:04:34.214399 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 19 11:04:34.214399 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 19 11:04:34.214399 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 19 11:04:34.214399 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 19 11:04:34.214399 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 19 11:04:34.214399 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 19 11:04:34.214399 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 19 11:04:34.214399 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 19 11:04:34.214399 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 19 11:04:34.214399 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 19 11:04:34.214399 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 19 11:04:34.215514 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 19 11:04:34.215514 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 19 11:04:34.215514 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 19 11:04:34.215514 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 19 11:04:34.215514 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume e71aa705-a579-4483-b8f5-b2f327acdbaf could not be found. Jul 19 11:04:34.215514 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:04:34.215514 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:04:34.215514 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:04:34.215514 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:04:34.215514 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:04:34.215514 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:04:34.215514 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:04:34.215514 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:04:34.215514 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 19 11:04:34.215514 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 19 11:04:34.215514 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:04:34.215514 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:04:34.215514 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Jul 19 11:04:34.215514 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:04:34.216512 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume e71aa705-a579-4483-b8f5-b2f327acdbaf could not be found. Jul 19 11:04:34.216512 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:04:34.216512 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:04:34.216512 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:04:34.216512 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:04:34.216512 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:04:34.216512 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:04:34.216512 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:04:34.216512 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:04:34.216512 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume e71aa705-a579-4483-b8f5-b2f327acdbaf could not be found. Jul 19 11:04:34.216512 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:04:34.216512 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-97ff25d0-9832-4896-94dc-3aa6c86896f6 tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] Returning 404 to user: Volume e71aa705-a579-4483-b8f5-b2f327acdbaf could not be found. {{(pid=117849) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:04:34.216512 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-97ff25d0-9832-4896-94dc-3aa6c86896f6 tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] 10.222.0.33 "DELETE /compute/v2.1/os-volumes/e71aa705-a579-4483-b8f5-b2f327acdbaf" status: 404 len: 109 microversion: 2.1 time: 0.110522 Jul 19 11:04:34.216512 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 22/35] 10.222.0.33 () {58 vars in 1247 bytes} [Tue Jul 19 11:04:34 2022] DELETE /compute/v2.1/os-volumes/e71aa705-a579-4483-b8f5-b2f327acdbaf => generated 109 bytes in 113 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 19 11:04:34.221779 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-4a7da635-6e68-49b1-a32d-02139d1746df tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] Calling method '.version_select at 0x7f04ac219700>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:34.379109 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.wsgi [None req-4a7da635-6e68-49b1-a32d-02139d1746df tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] HTTP exception thrown: Volume e71aa705-a579-4483-b8f5-b2f327acdbaf could not be found. Jul 19 11:04:34.381147 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-4a7da635-6e68-49b1-a32d-02139d1746df tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] Request method failure captured: Jul 19 11:04:34.381147 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: request: GET /compute/v2.1/os-volumes/e71aa705-a579-4483-b8f5-b2f327acdbaf HTTP/1.1 Jul 19 11:04:34.381147 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept: application/json Jul 19 11:04:34.381147 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept-Encoding: identity Jul 19 11:04:34.381147 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Connection: close Jul 19 11:04:34.381147 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Length: 0 Jul 19 11:04:34.381147 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Type: application/json Jul 19 11:04:34.381147 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Host: 10.222.0.33 Jul 19 11:04:34.381147 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Openstack-System-Scope: None Jul 19 11:04:34.381147 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: User-Agent: python-urllib3/1.26.9 Jul 19 11:04:34.381147 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Auth-Token: gAAAAABi1o-3Acst1r879RI9dFHt7UJWzPFghEd6G_jFMFMtxp3NKS13B-_BnHBn63S7pGqqoqx8J4I5JTifCCqB4Ih1J-wEPKLiQXmZOvOEqSvzdQOCkUbibasYuhFeMPZx3Vbxy2zpZM0DXFRSV8VGjxwthpDP54keDwSCrvQRs2RPv2cJRzA Jul 19 11:04:34.381147 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Id: None Jul 19 11:04:34.381147 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Name: None Jul 19 11:04:34.381147 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Identity-Status: Confirmed Jul 19 11:04:34.381147 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Is-Admin-Project: True Jul 19 11:04:34.381147 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Id: default Jul 19 11:04:34.381147 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Name: Default Jul 19 11:04:34.381147 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Id: 888be6c04e3e4248a000a1c601a1c8ed Jul 19 11:04:34.381147 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1903635009 Jul 19 11:04:34.381147 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Role: member,reader Jul 19 11:04:34.381147 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Roles: member,reader Jul 19 11:04:34.382732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/888be6c04e3e4248a000a1c601a1c8ed"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/888be6c04e3e4248a000a1c601a1c8ed"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/888be6c04e3e4248a000a1c601a1c8ed"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_888be6c04e3e4248a000a1c601a1c8ed"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:04:34.382732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1903635009 Jul 19 11:04:34.382732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Id: 888be6c04e3e4248a000a1c601a1c8ed Jul 19 11:04:34.382732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1903635009 Jul 19 11:04:34.382732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User: tempest-VolumesSnapshotsTestJSON-1903635009-project Jul 19 11:04:34.382732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Id: default Jul 19 11:04:34.382732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Name: Default Jul 19 11:04:34.382732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Id: 2c9c1fff948941b8baacac536fc09724 Jul 19 11:04:34.382732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1903635009-project Jul 19 11:04:34.382732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: method: Jul 19 11:04:34.382732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: exception: Volume e71aa705-a579-4483-b8f5-b2f327acdbaf could not be found. Jul 19 11:04:34.382732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: action_args: {'id': 'e71aa705-a579-4483-b8f5-b2f327acdbaf'} Jul 19 11:04:34.382732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:04:34.383377 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:04:34.383377 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 19 11:04:34.383377 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 19 11:04:34.383377 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 19 11:04:34.383377 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 19 11:04:34.383377 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 19 11:04:34.383377 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 19 11:04:34.383377 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 19 11:04:34.383377 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 19 11:04:34.383377 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 19 11:04:34.383377 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 19 11:04:34.383377 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 19 11:04:34.383377 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 19 11:04:34.383377 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 19 11:04:34.383377 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 19 11:04:34.383377 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume e71aa705-a579-4483-b8f5-b2f327acdbaf could not be found. (HTTP 404) (Request-ID: req-8c76bdc1-190a-4743-b42c-bba2e71d2be6) Jul 19 11:04:34.383377 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:04:34.383377 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:04:34.383377 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:04:34.384259 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:04:34.384259 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Jul 19 11:04:34.384259 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Jul 19 11:04:34.384259 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 19 11:04:34.384259 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 19 11:04:34.384259 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 19 11:04:34.384259 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 19 11:04:34.384259 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 19 11:04:34.384259 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 19 11:04:34.384259 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 19 11:04:34.384259 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 19 11:04:34.384259 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 19 11:04:34.384259 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 19 11:04:34.384259 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 19 11:04:34.384259 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 19 11:04:34.384259 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 19 11:04:34.384259 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 19 11:04:34.384259 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 19 11:04:34.384259 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 19 11:04:34.385165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 19 11:04:34.385165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 19 11:04:34.385165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 19 11:04:34.385165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 19 11:04:34.385165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume e71aa705-a579-4483-b8f5-b2f327acdbaf could not be found. Jul 19 11:04:34.385165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:04:34.385165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:04:34.385165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:04:34.385165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:04:34.385165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:04:34.385165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:04:34.385165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:04:34.385165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:04:34.385165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 19 11:04:34.385165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 19 11:04:34.385165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:04:34.385165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:04:34.385165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Jul 19 11:04:34.385165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:04:34.386030 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume e71aa705-a579-4483-b8f5-b2f327acdbaf could not be found. Jul 19 11:04:34.386030 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:04:34.386030 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:04:34.386030 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:04:34.386030 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:04:34.386030 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:04:34.386030 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:04:34.386030 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:04:34.386030 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:04:34.386030 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume e71aa705-a579-4483-b8f5-b2f327acdbaf could not be found. Jul 19 11:04:34.386030 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:04:34.386030 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-4a7da635-6e68-49b1-a32d-02139d1746df tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] Returning 404 to user: Volume e71aa705-a579-4483-b8f5-b2f327acdbaf could not be found. {{(pid=117849) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:04:34.386030 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-4a7da635-6e68-49b1-a32d-02139d1746df tempest-VolumesSnapshotsTestJSON-1903635009 tempest-VolumesSnapshotsTestJSON-1903635009-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/e71aa705-a579-4483-b8f5-b2f327acdbaf" status: 404 len: 109 microversion: 2.1 time: 0.165071 Jul 19 11:04:34.387168 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 23/36] 10.222.0.33 () {58 vars in 1244 bytes} [Tue Jul 19 11:04:34 2022] GET /compute/v2.1/os-volumes/e71aa705-a579-4483-b8f5-b2f327acdbaf => generated 109 bytes in 169 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:04:34.413860 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-1356d909-04f1-481b-bccb-fcd91e81e938 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-176927805", "imageRef": "3085883e-2289-4314-aa47-9c41d4368b02", "flavorRef": "452", "networks": [{"uuid": "7be74b19-036d-4488-a91f-a666900435da"}]}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:04:34.651936 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.network.neutron [None req-1356d909-04f1-481b-bccb-fcd91e81e938 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] validate_networks() for [('7be74b19-036d-4488-a91f-a666900435da', None, None, None, None, None)] {{(pid=117848) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 19 11:04:36.062460 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-2f29fcc6-0939-4aeb-97f5-fc9b56246f55 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1965301850", "imageRef": "3085883e-2289-4314-aa47-9c41d4368b02", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:04:36.270205 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.network.neutron [None req-2f29fcc6-0939-4aeb-97f5-fc9b56246f55 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] validate_networks() for [('1ffa3686-4e5a-4747-8994-8cc33db2a036', None, None, None, None, None)] {{(pid=117849) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 19 11:04:37.073211 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-1356d909-04f1-481b-bccb-fcd91e81e938 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Flavor limits 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 19 11:04:37.073825 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-1356d909-04f1-481b-bccb-fcd91e81e938 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Image limits 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 19 11:04:37.074638 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-1356d909-04f1-481b-bccb-fcd91e81e938 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Flavor pref 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 19 11:04:37.075617 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-1356d909-04f1-481b-bccb-fcd91e81e938 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Image pref 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 19 11:04:37.076286 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-1356d909-04f1-481b-bccb-fcd91e81e938 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 19 11:04:37.128675 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1356d909-04f1-481b-bccb-fcd91e81e938 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:37.132334 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1356d909-04f1-481b-bccb-fcd91e81e938 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:37.134723 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1356d909-04f1-481b-bccb-fcd91e81e938 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:37.137499 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1356d909-04f1-481b-bccb-fcd91e81e938 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:37.232600 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.quota [None req-1356d909-04f1-481b-bccb-fcd91e81e938 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Getting quotas for project 205683784b714ab399311aeca15d0cfa. Resources: {'cores', 'instances', 'ram'} {{(pid=117848) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:04:37.266584 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.quota [None req-1356d909-04f1-481b-bccb-fcd91e81e938 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Getting quotas for user f6cb55e4c41f4e6d94156c597a1baad8 and project 205683784b714ab399311aeca15d0cfa. Resources: {'cores', 'instances', 'ram'} {{(pid=117848) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:04:37.297432 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-1356d909-04f1-481b-bccb-fcd91e81e938 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Going to run 1 instances... {{(pid=117848) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 19 11:04:37.316001 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-1356d909-04f1-481b-bccb-fcd91e81e938 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='3085883e-2289-4314-aa47-9c41d4368b02',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117848) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 19 11:04:37.518424 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-1356d909-04f1-481b-bccb-fcd91e81e938 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 3.678063 Jul 19 11:04:37.529613 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 14/37] 10.222.0.33 () {60 vars in 1146 bytes} [Tue Jul 19 11:04:33 2022] POST /compute/v2.1/servers => generated 380 bytes in 3690 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 19 11:04:37.558672 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-91c2cd98-23c8-4670-9bbd-6d95c3ab03cb tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:37.560024 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-91c2cd98-23c8-4670-9bbd-6d95c3ab03cb tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:04:38.554136 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-91c2cd98-23c8-4670-9bbd-6d95c3ab03cb tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:38.621072 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-91c2cd98-23c8-4670-9bbd-6d95c3ab03cb tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1214 microversion: 2.1 time: 1.077826 Jul 19 11:04:38.622000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 15/38] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:04:37 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1214 bytes in 1092 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:04:39.504619 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-2f29fcc6-0939-4aeb-97f5-fc9b56246f55 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Flavor limits 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 19 11:04:39.510267 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-2f29fcc6-0939-4aeb-97f5-fc9b56246f55 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Image limits 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 19 11:04:39.511071 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-2f29fcc6-0939-4aeb-97f5-fc9b56246f55 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Flavor pref 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 19 11:04:39.511687 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-2f29fcc6-0939-4aeb-97f5-fc9b56246f55 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Image pref 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 19 11:04:39.513079 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-2f29fcc6-0939-4aeb-97f5-fc9b56246f55 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 19 11:04:39.557995 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2f29fcc6-0939-4aeb-97f5-fc9b56246f55 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:39.560439 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2f29fcc6-0939-4aeb-97f5-fc9b56246f55 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:39.575209 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2f29fcc6-0939-4aeb-97f5-fc9b56246f55 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:39.575798 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2f29fcc6-0939-4aeb-97f5-fc9b56246f55 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:39.661599 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-2f29fcc6-0939-4aeb-97f5-fc9b56246f55 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Getting quotas for project 05fceba2c70c40a399e71daabe8edef6. Resources: {'cores', 'instances', 'ram'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:04:39.682725 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-2f29fcc6-0939-4aeb-97f5-fc9b56246f55 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Getting quotas for user c95281b28eae450d8678583a929c3b8a and project 05fceba2c70c40a399e71daabe8edef6. Resources: {'cores', 'instances', 'ram'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:04:39.737998 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-2f29fcc6-0939-4aeb-97f5-fc9b56246f55 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Going to run 1 instances... {{(pid=117849) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 19 11:04:39.774038 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-2f29fcc6-0939-4aeb-97f5-fc9b56246f55 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='3085883e-2289-4314-aa47-9c41d4368b02',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117849) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 19 11:04:40.021734 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-2f29fcc6-0939-4aeb-97f5-fc9b56246f55 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 4.285826 Jul 19 11:04:40.024295 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 24/39] 10.222.0.33 () {60 vars in 1146 bytes} [Tue Jul 19 11:04:35 2022] POST /compute/v2.1/servers => generated 383 bytes in 4289 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 19 11:04:40.041811 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-ed84af14-b3d7-4d92-a329-4bd1ecf08f1f tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:40.047962 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-ed84af14-b3d7-4d92-a329-4bd1ecf08f1f tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:04:40.214399 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-ed84af14-b3d7-4d92-a329-4bd1ecf08f1f tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:40.231958 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-ed84af14-b3d7-4d92-a329-4bd1ecf08f1f tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1214 microversion: 2.1 time: 0.204291 Jul 19 11:04:40.236135 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 25/40] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:04:40 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1214 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:04:40.557663 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-0f024d37-5904-4bc6-8631-e013be79922c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Action: 'create', calling method: .version_select at 0x7f04ac1f0a60>, body: {"keypair": {"name": "tempest-keypair-730599398", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMQmy9znFTse42bAxe+FrnVQBd19WNR1fpMYwXpS9GmvYub51ISKVla2OGT8pOQxXTOknh2fUam9l6mQG1iiyNYQkcsQp/CAENfBx5UX3J1kvNAFvVlKQS8x69v8D+qFpA=="}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:04:40.653348 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.quota [None req-0f024d37-5904-4bc6-8631-e013be79922c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Getting quotas for project e1d59596ecae4e288733b90c577aa86b. Resources: {'key_pairs'} {{(pid=117848) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:04:40.673660 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.quota [None req-0f024d37-5904-4bc6-8631-e013be79922c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Getting quotas for user 246dc188d9a44b1cb38721e3dd8a863a and project e1d59596ecae4e288733b90c577aa86b. Resources: {'key_pairs'} {{(pid=117848) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:04:40.810442 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-0f024d37-5904-4bc6-8631-e013be79922c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 1.238132 Jul 19 11:04:40.818568 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 16/41] 10.222.0.33 () {62 vars in 1199 bytes} [Tue Jul 19 11:04:39 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 1241 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Jul 19 11:04:40.823193 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-186b0222-4ff1-4ad3-8a92-0f481c0b535d tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:40.834746 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-186b0222-4ff1-4ad3-8a92-0f481c0b535d tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:04:40.948808 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-186b0222-4ff1-4ad3-8a92-0f481c0b535d tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:40.961323 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-186b0222-4ff1-4ad3-8a92-0f481c0b535d tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1222 microversion: 2.1 time: 0.145917 Jul 19 11:04:40.963180 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 17/42] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:04:40 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1222 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:04:41.013572 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-e8418368-e1c6-4812-8afb-3ef8eaf6d470 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Action: 'create', calling method: .version_select at 0x7f04ab884820>, body: {"keypair": {"name": "tempest-keypair-1514182937", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBG+KGFtVg98U5gR6Pce+vCqRLXMIHFYvmqvYZeL7csW8l15mht87SUEcQPLA3MhfMz6/7bTKl0hcAoYZCJz7eS4fMKM7hiqGPgN44Rey9gGe+PJGKlhuZ2LMKVdgQc89Dg=="}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:04:41.066955 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-e8418368-e1c6-4812-8afb-3ef8eaf6d470 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Getting quotas for project ef534f1d06ea415d98857ec7f9e5fb97. Resources: {'key_pairs'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:04:41.076994 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-e8418368-e1c6-4812-8afb-3ef8eaf6d470 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Getting quotas for user b8f2a78f3a3e4b8e8937e3bc39524690 and project ef534f1d06ea415d98857ec7f9e5fb97. Resources: {'key_pairs'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:04:41.253189 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-d9d96023-4124-4046-9b4d-57fbb99ac52b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-757440931", "imageRef": "3085883e-2289-4314-aa47-9c41d4368b02", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:04:41.259999 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-e8418368-e1c6-4812-8afb-3ef8eaf6d470 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 1.021747 Jul 19 11:04:41.260619 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 26/43] 10.222.0.33 () {60 vars in 1158 bytes} [Tue Jul 19 11:04:40 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 1023 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:04:41.285965 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-0cbf2ea3-643f-4349-bbe9-46d1a4e563e2 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:41.286713 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-0cbf2ea3-643f-4349-bbe9-46d1a4e563e2 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:04:41.311913 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0cbf2ea3-643f-4349-bbe9-46d1a4e563e2 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:41.312747 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0cbf2ea3-643f-4349-bbe9-46d1a4e563e2 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:41.316311 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0cbf2ea3-643f-4349-bbe9-46d1a4e563e2 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:41.317081 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0cbf2ea3-643f-4349-bbe9-46d1a4e563e2 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:41.378043 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.network.neutron [None req-d9d96023-4124-4046-9b4d-57fbb99ac52b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] validate_networks() for [('1ffa3686-4e5a-4747-8994-8cc33db2a036', None, None, None, None, None)] {{(pid=117848) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 19 11:04:41.910202 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-0cbf2ea3-643f-4349-bbe9-46d1a4e563e2 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:41.939723 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-0cbf2ea3-643f-4349-bbe9-46d1a4e563e2 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1210 microversion: 2.1 time: 0.677976 Jul 19 11:04:41.955096 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 27/44] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:04:41 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1210 bytes in 694 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:04:42.634425 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-d9d96023-4124-4046-9b4d-57fbb99ac52b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Flavor limits 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 19 11:04:42.635734 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-d9d96023-4124-4046-9b4d-57fbb99ac52b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Image limits 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 19 11:04:42.636434 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-d9d96023-4124-4046-9b4d-57fbb99ac52b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Flavor pref 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 19 11:04:42.637176 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-d9d96023-4124-4046-9b4d-57fbb99ac52b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Image pref 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 19 11:04:42.637807 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-d9d96023-4124-4046-9b4d-57fbb99ac52b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 19 11:04:42.663776 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d9d96023-4124-4046-9b4d-57fbb99ac52b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:42.664409 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d9d96023-4124-4046-9b4d-57fbb99ac52b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:42.669543 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d9d96023-4124-4046-9b4d-57fbb99ac52b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:42.670354 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d9d96023-4124-4046-9b4d-57fbb99ac52b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:42.727607 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.quota [None req-d9d96023-4124-4046-9b4d-57fbb99ac52b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Getting quotas for project 05fceba2c70c40a399e71daabe8edef6. Resources: {'cores', 'instances', 'ram'} {{(pid=117848) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:04:42.737756 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.quota [None req-d9d96023-4124-4046-9b4d-57fbb99ac52b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Getting quotas for user c95281b28eae450d8678583a929c3b8a and project 05fceba2c70c40a399e71daabe8edef6. Resources: {'cores', 'instances', 'ram'} {{(pid=117848) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:04:42.766986 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-d9d96023-4124-4046-9b4d-57fbb99ac52b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Going to run 1 instances... {{(pid=117848) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 19 11:04:42.781398 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-d9d96023-4124-4046-9b4d-57fbb99ac52b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='3085883e-2289-4314-aa47-9c41d4368b02',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117848) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 19 11:04:42.875609 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-d9d96023-4124-4046-9b4d-57fbb99ac52b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.630843 Jul 19 11:04:42.882596 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 18/45] 10.222.0.33 () {60 vars in 1146 bytes} [Tue Jul 19 11:04:41 2022] POST /compute/v2.1/servers => generated 383 bytes in 1638 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 19 11:04:42.887985 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-dfc47912-f6ab-4a41-b72b-ba5af1317b49 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:42.889818 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-dfc47912-f6ab-4a41-b72b-ba5af1317b49 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:04:42.962315 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-db21f914-a119-4ec3-af5b-f6ea9cfe1eb4 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:42.968924 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-db21f914-a119-4ec3-af5b-f6ea9cfe1eb4 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:04:42.986995 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-db21f914-a119-4ec3-af5b-f6ea9cfe1eb4 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:42.989795 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-db21f914-a119-4ec3-af5b-f6ea9cfe1eb4 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:42.993627 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-db21f914-a119-4ec3-af5b-f6ea9cfe1eb4 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:42.997336 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-db21f914-a119-4ec3-af5b-f6ea9cfe1eb4 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:43.556476 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-dfc47912-f6ab-4a41-b72b-ba5af1317b49 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:43.602797 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-dfc47912-f6ab-4a41-b72b-ba5af1317b49 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1221 microversion: 2.1 time: 0.701987 Jul 19 11:04:43.603466 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 28/46] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:04:42 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1221 bytes in 721 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:04:43.709456 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-db21f914-a119-4ec3-af5b-f6ea9cfe1eb4 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:43.739577 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-db21f914-a119-4ec3-af5b-f6ea9cfe1eb4 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1272 microversion: 2.1 time: 0.781814 Jul 19 11:04:43.740357 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 19/47] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:04:42 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1272 bytes in 783 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:04:44.619866 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-f77b3fad-f22b-449b-a1ee-3157f5017010 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:44.620488 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-f77b3fad-f22b-449b-a1ee-3157f5017010 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:04:44.658932 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f77b3fad-f22b-449b-a1ee-3157f5017010 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:44.661037 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f77b3fad-f22b-449b-a1ee-3157f5017010 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:44.665899 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f77b3fad-f22b-449b-a1ee-3157f5017010 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:44.668848 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f77b3fad-f22b-449b-a1ee-3157f5017010 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:44.759117 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-d0562756-51e7-41f3-a057-c669445939e6 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:44.760473 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-d0562756-51e7-41f3-a057-c669445939e6 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:04:44.788081 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d0562756-51e7-41f3-a057-c669445939e6 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:44.788628 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d0562756-51e7-41f3-a057-c669445939e6 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:44.793792 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d0562756-51e7-41f3-a057-c669445939e6 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:44.797154 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d0562756-51e7-41f3-a057-c669445939e6 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:45.106588 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-d0562756-51e7-41f3-a057-c669445939e6 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:45.119355 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-d0562756-51e7-41f3-a057-c669445939e6 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1272 microversion: 2.1 time: 0.364980 Jul 19 11:04:45.124076 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 20/48] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:04:44 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1272 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:04:45.318635 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-f77b3fad-f22b-449b-a1ee-3157f5017010 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:45.328133 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-f77b3fad-f22b-449b-a1ee-3157f5017010 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1273 microversion: 2.1 time: 0.712351 Jul 19 11:04:45.354764 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 29/49] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:04:44 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1273 bytes in 739 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:04:46.143537 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-787b9252-9b2a-413b-a793-91091e1baf1b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:46.146255 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-787b9252-9b2a-413b-a793-91091e1baf1b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:04:46.163235 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-787b9252-9b2a-413b-a793-91091e1baf1b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:46.164216 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-787b9252-9b2a-413b-a793-91091e1baf1b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:46.169899 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-787b9252-9b2a-413b-a793-91091e1baf1b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:46.172760 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-787b9252-9b2a-413b-a793-91091e1baf1b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:46.354588 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-21d49a85-a900-4870-b42a-e50e86940260 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:46.357550 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-21d49a85-a900-4870-b42a-e50e86940260 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:04:46.381252 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-21d49a85-a900-4870-b42a-e50e86940260 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:46.381780 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-21d49a85-a900-4870-b42a-e50e86940260 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:46.390725 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-21d49a85-a900-4870-b42a-e50e86940260 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:46.399950 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-21d49a85-a900-4870-b42a-e50e86940260 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:46.999791 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-787b9252-9b2a-413b-a793-91091e1baf1b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:47.008878 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-787b9252-9b2a-413b-a793-91091e1baf1b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1272 microversion: 2.1 time: 0.873400 Jul 19 11:04:47.009468 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 21/50] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:04:46 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1272 bytes in 875 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:04:47.108327 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-21d49a85-a900-4870-b42a-e50e86940260 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:47.133913 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-21d49a85-a900-4870-b42a-e50e86940260 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1279 microversion: 2.1 time: 0.785975 Jul 19 11:04:47.148322 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 30/51] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:04:46 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1279 bytes in 802 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:04:48.027170 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-5115235d-0324-4bdb-b521-aaaa52d55c2c tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:48.027792 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-5115235d-0324-4bdb-b521-aaaa52d55c2c tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:04:48.044464 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5115235d-0324-4bdb-b521-aaaa52d55c2c tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:48.044969 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5115235d-0324-4bdb-b521-aaaa52d55c2c tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:48.052011 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5115235d-0324-4bdb-b521-aaaa52d55c2c tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:48.055226 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5115235d-0324-4bdb-b521-aaaa52d55c2c tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:48.165669 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-87969645-b497-4c13-8178-15a75501a156 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:48.167869 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-87969645-b497-4c13-8178-15a75501a156 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:04:48.174922 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-87969645-b497-4c13-8178-15a75501a156 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:48.180951 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-87969645-b497-4c13-8178-15a75501a156 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:48.185581 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-87969645-b497-4c13-8178-15a75501a156 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:48.188484 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-87969645-b497-4c13-8178-15a75501a156 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:48.234922 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-5115235d-0324-4bdb-b521-aaaa52d55c2c tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:48.245935 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-5115235d-0324-4bdb-b521-aaaa52d55c2c tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1272 microversion: 2.1 time: 0.223291 Jul 19 11:04:48.251137 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 22/52] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:04:48 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1272 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:04:48.257865 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-a759c787-79ad-4605-b965-b025c381832d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-870457946", "imageRef": "3085883e-2289-4314-aa47-9c41d4368b02", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "4e7c735a-1e35-464a-9187-a0d8149bbb40"}], "security_groups": [{"name": "tempest-securitygroup--939973465"}], "key_name": "tempest-keypair-730599398", "user_data": "}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:04:48.356619 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-87969645-b497-4c13-8178-15a75501a156 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:48.366468 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-87969645-b497-4c13-8178-15a75501a156 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1279 microversion: 2.1 time: 0.205470 Jul 19 11:04:48.378055 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 31/53] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:04:48 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1279 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:04:48.823682 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.network.neutron [None req-a759c787-79ad-4605-b965-b025c381832d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] validate_networks() for [('4e7c735a-1e35-464a-9187-a0d8149bbb40', None, None, None, None, None)] {{(pid=117848) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 19 11:04:49.269213 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-506d6589-d6a3-4c80-8c73-b10cb01e8cc4 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:49.270394 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-506d6589-d6a3-4c80-8c73-b10cb01e8cc4 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:04:49.293452 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-506d6589-d6a3-4c80-8c73-b10cb01e8cc4 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:49.294136 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-506d6589-d6a3-4c80-8c73-b10cb01e8cc4 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:49.295548 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-506d6589-d6a3-4c80-8c73-b10cb01e8cc4 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:49.296377 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-506d6589-d6a3-4c80-8c73-b10cb01e8cc4 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:49.496941 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-506d6589-d6a3-4c80-8c73-b10cb01e8cc4 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:49.505371 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-506d6589-d6a3-4c80-8c73-b10cb01e8cc4 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1272 microversion: 2.1 time: 0.241809 Jul 19 11:04:49.535756 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 32/54] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:04:49 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1272 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:04:49.572959 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-b4b2760d-c418-4a92-9ddf-bcfa824533ca tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:49.578432 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-b4b2760d-c418-4a92-9ddf-bcfa824533ca tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:04:49.602486 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b4b2760d-c418-4a92-9ddf-bcfa824533ca tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:49.602968 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b4b2760d-c418-4a92-9ddf-bcfa824533ca tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:49.607453 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b4b2760d-c418-4a92-9ddf-bcfa824533ca tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:49.609929 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b4b2760d-c418-4a92-9ddf-bcfa824533ca tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:49.619054 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell0:SELECT=2 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:04:50.214866 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b4b2760d-c418-4a92-9ddf-bcfa824533ca tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:50.225566 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-b4b2760d-c418-4a92-9ddf-bcfa824533ca tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1279 microversion: 2.1 time: 0.666436 Jul 19 11:04:50.231503 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 33/55] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:04:49 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1279 bytes in 695 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:04:50.539294 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-ae93bf25-e719-4b35-91b6-8d4bbfc60537 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:50.540618 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-ae93bf25-e719-4b35-91b6-8d4bbfc60537 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:04:50.549955 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ae93bf25-e719-4b35-91b6-8d4bbfc60537 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:50.551732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ae93bf25-e719-4b35-91b6-8d4bbfc60537 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:50.558591 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-a759c787-79ad-4605-b965-b025c381832d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Flavor limits 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 19 11:04:50.560523 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-a759c787-79ad-4605-b965-b025c381832d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Image limits 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 19 11:04:50.561547 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-a759c787-79ad-4605-b965-b025c381832d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Flavor pref 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 19 11:04:50.562032 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-a759c787-79ad-4605-b965-b025c381832d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Image pref 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 19 11:04:50.562598 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-a759c787-79ad-4605-b965-b025c381832d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 19 11:04:50.572672 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ae93bf25-e719-4b35-91b6-8d4bbfc60537 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:50.574910 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ae93bf25-e719-4b35-91b6-8d4bbfc60537 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.019s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:50.585174 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a759c787-79ad-4605-b965-b025c381832d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:50.585626 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a759c787-79ad-4605-b965-b025c381832d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:50.586428 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a759c787-79ad-4605-b965-b025c381832d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:50.586867 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a759c787-79ad-4605-b965-b025c381832d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:50.681258 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.quota [None req-a759c787-79ad-4605-b965-b025c381832d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Getting quotas for project e1d59596ecae4e288733b90c577aa86b. Resources: {'cores', 'instances', 'ram'} {{(pid=117848) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:04:50.695225 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.quota [None req-a759c787-79ad-4605-b965-b025c381832d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Getting quotas for user 246dc188d9a44b1cb38721e3dd8a863a and project e1d59596ecae4e288733b90c577aa86b. Resources: {'cores', 'instances', 'ram'} {{(pid=117848) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:04:50.721083 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-a759c787-79ad-4605-b965-b025c381832d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Going to run 1 instances... {{(pid=117848) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 19 11:04:50.724182 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-a759c787-79ad-4605-b965-b025c381832d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='3085883e-2289-4314-aa47-9c41d4368b02',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117848) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 19 11:04:50.811557 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-a759c787-79ad-4605-b965-b025c381832d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.20 time: 2.557737 Jul 19 11:04:50.824569 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 23/56] 10.222.0.33 () {62 vars in 1187 bytes} [Tue Jul 19 11:04:48 2022] POST /compute/v2.1/servers => generated 401 bytes in 2572 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Jul 19 11:04:50.830374 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-6201c249-c855-485c-b988-7d83b2b3837c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:50.831341 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-6201c249-c855-485c-b988-7d83b2b3837c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:04:50.952729 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-6201c249-c855-485c-b988-7d83b2b3837c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:50.974556 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-6201c249-c855-485c-b988-7d83b2b3837c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:50.978033 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-6201c249-c855-485c-b988-7d83b2b3837c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:50.979388 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-6201c249-c855-485c-b988-7d83b2b3837c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1280 microversion: 2.20 time: 0.153452 Jul 19 11:04:51.006528 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 24/57] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:04:50 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1280 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:04:51.255139 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-e62b87ea-5ab1-42a2-80cd-fef1bc00f3d6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:51.255812 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-e62b87ea-5ab1-42a2-80cd-fef1bc00f3d6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:04:51.292432 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e62b87ea-5ab1-42a2-80cd-fef1bc00f3d6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:51.306676 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e62b87ea-5ab1-42a2-80cd-fef1bc00f3d6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:51.307802 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e62b87ea-5ab1-42a2-80cd-fef1bc00f3d6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:51.308357 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e62b87ea-5ab1-42a2-80cd-fef1bc00f3d6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:52.276426 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-e62b87ea-5ab1-42a2-80cd-fef1bc00f3d6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:52.294506 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-e62b87ea-5ab1-42a2-80cd-fef1bc00f3d6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 1.054744 Jul 19 11:04:52.316012 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 25/58] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:04:51 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 1077 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:04:52.343781 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-f8957680-0f75-4a24-81b7-b6d80029586e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:52.346542 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-f8957680-0f75-4a24-81b7-b6d80029586e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:04:52.375560 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f8957680-0f75-4a24-81b7-b6d80029586e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:52.386746 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f8957680-0f75-4a24-81b7-b6d80029586e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:52.388131 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f8957680-0f75-4a24-81b7-b6d80029586e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:52.391740 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f8957680-0f75-4a24-81b7-b6d80029586e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:53.088648 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-f8957680-0f75-4a24-81b7-b6d80029586e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:53.097937 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-f8957680-0f75-4a24-81b7-b6d80029586e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:53.104080 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-f8957680-0f75-4a24-81b7-b6d80029586e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:53.108148 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-f8957680-0f75-4a24-81b7-b6d80029586e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1284 microversion: 2.20 time: 0.790766 Jul 19 11:04:53.113286 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 26/59] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:04:52 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1284 bytes in 796 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:04:53.327646 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-34ff319d-9dcd-4512-be7f-093ddbcfa949 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:53.329125 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-34ff319d-9dcd-4512-be7f-093ddbcfa949 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:04:53.340788 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-34ff319d-9dcd-4512-be7f-093ddbcfa949 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:53.347711 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-34ff319d-9dcd-4512-be7f-093ddbcfa949 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:53.348879 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-34ff319d-9dcd-4512-be7f-093ddbcfa949 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:53.349732 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-34ff319d-9dcd-4512-be7f-093ddbcfa949 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:53.508341 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-ae93bf25-e719-4b35-91b6-8d4bbfc60537 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:53.533960 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-ae93bf25-e719-4b35-91b6-8d4bbfc60537 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1314 microversion: 2.1 time: 2.988232 Jul 19 11:04:53.536600 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 34/60] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:04:50 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1314 bytes in 3004 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:04:54.132384 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-8fc02807-e230-4835-832d-8eb3f0194af8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:54.133357 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-8fc02807-e230-4835-832d-8eb3f0194af8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:04:54.135340 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell0:SELECT=14 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:04:54.147427 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8fc02807-e230-4835-832d-8eb3f0194af8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:54.168768 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8fc02807-e230-4835-832d-8eb3f0194af8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.021s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:54.174136 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8fc02807-e230-4835-832d-8eb3f0194af8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:54.184430 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8fc02807-e230-4835-832d-8eb3f0194af8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:54.520127 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-34ff319d-9dcd-4512-be7f-093ddbcfa949 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:54.543510 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-34ff319d-9dcd-4512-be7f-093ddbcfa949 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 1.224777 Jul 19 11:04:54.545141 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 27/61] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:04:53 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 1227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:04:54.554855 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-3b74adc8-5c00-4ac8-abd2-203e196c9f16 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:54.557775 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-3b74adc8-5c00-4ac8-abd2-203e196c9f16 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:04:54.580887 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3b74adc8-5c00-4ac8-abd2-203e196c9f16 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:54.581367 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3b74adc8-5c00-4ac8-abd2-203e196c9f16 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:54.582206 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3b74adc8-5c00-4ac8-abd2-203e196c9f16 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:54.585848 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3b74adc8-5c00-4ac8-abd2-203e196c9f16 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:55.040525 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-8fc02807-e230-4835-832d-8eb3f0194af8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:55.066142 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-8fc02807-e230-4835-832d-8eb3f0194af8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:55.081148 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-8fc02807-e230-4835-832d-8eb3f0194af8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:55.082931 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-8fc02807-e230-4835-832d-8eb3f0194af8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1340 microversion: 2.20 time: 0.955391 Jul 19 11:04:55.083916 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 35/62] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:04:54 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1340 bytes in 957 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:04:55.317293 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-3b74adc8-5c00-4ac8-abd2-203e196c9f16 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:55.325832 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-3b74adc8-5c00-4ac8-abd2-203e196c9f16 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1314 microversion: 2.1 time: 0.776897 Jul 19 11:04:55.338901 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 28/63] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:04:54 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1314 bytes in 790 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:04:55.560171 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-6b544505-6e00-4d39-b0d8-1994d251aa26 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:55.561043 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-6b544505-6e00-4d39-b0d8-1994d251aa26 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:04:55.585370 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6b544505-6e00-4d39-b0d8-1994d251aa26 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:55.586145 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6b544505-6e00-4d39-b0d8-1994d251aa26 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:55.589407 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6b544505-6e00-4d39-b0d8-1994d251aa26 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:55.590403 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6b544505-6e00-4d39-b0d8-1994d251aa26 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:55.925784 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-191e7a89-cf7a-4b17-a874-5c2eae90234c req-c6916fb2-9269-4410-9dea-c5e2580780cb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "49e783c8-8e6d-4789-b81d-8e1ca4a550a6", "tag": "ba1dd015-6965-43ea-a7cf-059d7c358686"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:04:55.946693 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-191e7a89-cf7a-4b17-a874-5c2eae90234c req-c6916fb2-9269-4410-9dea-c5e2580780cb service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:55.947151 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-191e7a89-cf7a-4b17-a874-5c2eae90234c req-c6916fb2-9269-4410-9dea-c5e2580780cb service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:55.969742 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-191e7a89-cf7a-4b17-a874-5c2eae90234c req-c6916fb2-9269-4410-9dea-c5e2580780cb service nova] Creating event network-changed:ba1dd015-6965-43ea-a7cf-059d7c358686 for instance 49e783c8-8e6d-4789-b81d-8e1ca4a550a6 on hv-cinder-maste Jul 19 11:04:55.970528 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-191e7a89-cf7a-4b17-a874-5c2eae90234c req-c6916fb2-9269-4410-9dea-c5e2580780cb service nova] Acquired lock "compute-rpcapi-router" {{(pid=117848) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} Jul 19 11:04:55.971794 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b68cb902-7599-414c-8035-13c0ed90cbb3 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:55.972388 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b68cb902-7599-414c-8035-13c0ed90cbb3 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:55.973613 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b68cb902-7599-414c-8035-13c0ed90cbb3 None None] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:55.974417 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b68cb902-7599-414c-8035-13c0ed90cbb3 None None] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:55.985082 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.compute.rpcapi [None req-b68cb902-7599-414c-8035-13c0ed90cbb3 None None] Automatically selected compute RPC version 6.0 from minimum service version 61 Jul 19 11:04:55.986280 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b68cb902-7599-414c-8035-13c0ed90cbb3 None None] Releasing lock "compute-rpcapi-router" {{(pid=117848) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Jul 19 11:04:56.012463 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-b68cb902-7599-414c-8035-13c0ed90cbb3 None None] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.193514 Jul 19 11:04:56.014353 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 29/64] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:04:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:04:56.109531 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-ba7d5507-22dc-4bc2-9d4b-0d82ec2ef574 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:56.110800 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-ba7d5507-22dc-4bc2-9d4b-0d82ec2ef574 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:04:56.120506 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ba7d5507-22dc-4bc2-9d4b-0d82ec2ef574 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:56.121267 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ba7d5507-22dc-4bc2-9d4b-0d82ec2ef574 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:56.122884 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ba7d5507-22dc-4bc2-9d4b-0d82ec2ef574 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:56.123993 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ba7d5507-22dc-4bc2-9d4b-0d82ec2ef574 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:56.135703 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-6b544505-6e00-4d39-b0d8-1994d251aa26 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:56.146960 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-6b544505-6e00-4d39-b0d8-1994d251aa26 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 0.590552 Jul 19 11:04:56.148212 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 36/65] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:04:55 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 593 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:04:56.211110 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-ba7d5507-22dc-4bc2-9d4b-0d82ec2ef574 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:56.220566 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-ba7d5507-22dc-4bc2-9d4b-0d82ec2ef574 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:56.222623 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-ba7d5507-22dc-4bc2-9d4b-0d82ec2ef574 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:56.224085 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-ba7d5507-22dc-4bc2-9d4b-0d82ec2ef574 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1338 microversion: 2.20 time: 0.124614 Jul 19 11:04:56.225354 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 30/66] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:04:56 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1338 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:04:56.356138 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-727cf8a3-73a2-4dfb-a6cd-9071fb30bd67 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:56.356895 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-727cf8a3-73a2-4dfb-a6cd-9071fb30bd67 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:04:56.374779 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-727cf8a3-73a2-4dfb-a6cd-9071fb30bd67 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:56.377319 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-727cf8a3-73a2-4dfb-a6cd-9071fb30bd67 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:56.378298 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-727cf8a3-73a2-4dfb-a6cd-9071fb30bd67 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:56.378812 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-727cf8a3-73a2-4dfb-a6cd-9071fb30bd67 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:56.667242 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-727cf8a3-73a2-4dfb-a6cd-9071fb30bd67 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:56.675998 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-727cf8a3-73a2-4dfb-a6cd-9071fb30bd67 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1314 microversion: 2.1 time: 0.324448 Jul 19 11:04:56.685186 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 37/67] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:04:56 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1314 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:04:56.762748 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-8dc08d29-3597-4f2b-9320-62241464bec7 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-660834346", "imageRef": "3085883e-2289-4314-aa47-9c41d4368b02", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "0b1fabad-b81a-4ee7-9fd9-62393ba0d37f"}], "security_groups": [{"name": "tempest-securitygroup--1368874103"}], "key_name": "tempest-keypair-1514182937", "user_data": "}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:04:56.926439 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-94e6b143-0254-4f84-9c5c-d36f87c2c679 req-a26df7d5-fbb6-4eb2-84ba-74abc9851e97 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "126c2860-81b1-42fc-8fbd-78178837f2c2", "tag": "ab2b6732-5d45-4b40-8888-44a729c6870f"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:04:56.952017 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-94e6b143-0254-4f84-9c5c-d36f87c2c679 req-a26df7d5-fbb6-4eb2-84ba-74abc9851e97 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:56.957971 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-94e6b143-0254-4f84-9c5c-d36f87c2c679 req-a26df7d5-fbb6-4eb2-84ba-74abc9851e97 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:57.011253 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-94e6b143-0254-4f84-9c5c-d36f87c2c679 req-a26df7d5-fbb6-4eb2-84ba-74abc9851e97 service nova] Creating event network-changed:ab2b6732-5d45-4b40-8888-44a729c6870f for instance 126c2860-81b1-42fc-8fbd-78178837f2c2 on hv-cinder-maste Jul 19 11:04:57.040450 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-94e6b143-0254-4f84-9c5c-d36f87c2c679 req-a26df7d5-fbb6-4eb2-84ba-74abc9851e97 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.360674 Jul 19 11:04:57.046223 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 31/68] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:04:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:04:57.131003 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.network.neutron [None req-8dc08d29-3597-4f2b-9320-62241464bec7 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] validate_networks() for [('0b1fabad-b81a-4ee7-9fd9-62393ba0d37f', None, None, None, None, None)] {{(pid=117849) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 19 11:04:57.165580 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-42fd8c53-25f6-4eef-bfc5-2f5aad2fede7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:57.166588 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-42fd8c53-25f6-4eef-bfc5-2f5aad2fede7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:04:57.184671 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-42fd8c53-25f6-4eef-bfc5-2f5aad2fede7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:57.187447 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-42fd8c53-25f6-4eef-bfc5-2f5aad2fede7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:57.188535 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-42fd8c53-25f6-4eef-bfc5-2f5aad2fede7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:57.191008 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-42fd8c53-25f6-4eef-bfc5-2f5aad2fede7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:57.458149 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-42fd8c53-25f6-4eef-bfc5-2f5aad2fede7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:57.470268 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-42fd8c53-25f6-4eef-bfc5-2f5aad2fede7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 0.309497 Jul 19 11:04:57.471877 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 32/69] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:04:57 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:04:57.479017 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-dfaf79b1-d4c2-4601-8f94-9fee804e6bd9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:57.479856 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-dfaf79b1-d4c2-4601-8f94-9fee804e6bd9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:04:57.483203 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_api:SELECT=90,nova_api:INSERT=10 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:04:57.488228 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dfaf79b1-d4c2-4601-8f94-9fee804e6bd9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:57.489004 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dfaf79b1-d4c2-4601-8f94-9fee804e6bd9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:57.489978 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dfaf79b1-d4c2-4601-8f94-9fee804e6bd9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:57.490688 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dfaf79b1-d4c2-4601-8f94-9fee804e6bd9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:58.106246 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-dfaf79b1-d4c2-4601-8f94-9fee804e6bd9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:58.119810 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-dfaf79b1-d4c2-4601-8f94-9fee804e6bd9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:58.124289 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-dfaf79b1-d4c2-4601-8f94-9fee804e6bd9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:58.125618 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-dfaf79b1-d4c2-4601-8f94-9fee804e6bd9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1338 microversion: 2.20 time: 0.652166 Jul 19 11:04:58.133175 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 33/70] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:04:57 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1338 bytes in 660 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:04:58.149218 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-84c52c35-30bd-4a6c-bf7a-3cd306fda604 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:58.150861 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-84c52c35-30bd-4a6c-bf7a-3cd306fda604 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:04:58.166053 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-84c52c35-30bd-4a6c-bf7a-3cd306fda604 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:58.169414 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-84c52c35-30bd-4a6c-bf7a-3cd306fda604 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:58.170831 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-84c52c35-30bd-4a6c-bf7a-3cd306fda604 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:58.173398 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-84c52c35-30bd-4a6c-bf7a-3cd306fda604 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:59.394622 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-84c52c35-30bd-4a6c-bf7a-3cd306fda604 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:04:59.404793 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-84c52c35-30bd-4a6c-bf7a-3cd306fda604 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1314 microversion: 2.1 time: 1.270033 Jul 19 11:04:59.410038 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 34/71] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:04:58 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1314 bytes in 1276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:04:59.415912 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-de7fbf5d-c282-4e38-a5c2-a63d434fb744 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:04:59.422026 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-de7fbf5d-c282-4e38-a5c2-a63d434fb744 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:04:59.436026 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-de7fbf5d-c282-4e38-a5c2-a63d434fb744 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:59.441188 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-de7fbf5d-c282-4e38-a5c2-a63d434fb744 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:04:59.442535 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-de7fbf5d-c282-4e38-a5c2-a63d434fb744 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:04:59.443253 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-de7fbf5d-c282-4e38-a5c2-a63d434fb744 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:00.143949 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-de7fbf5d-c282-4e38-a5c2-a63d434fb744 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:00.154897 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-de7fbf5d-c282-4e38-a5c2-a63d434fb744 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 0.741479 Jul 19 11:05:00.164209 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 35/72] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:04:59 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 746 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:00.166285 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-ba2ad517-aad5-4175-b444-9a016c517dc5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:00.172577 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-ba2ad517-aad5-4175-b444-9a016c517dc5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:00.188852 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ba2ad517-aad5-4175-b444-9a016c517dc5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:00.193123 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ba2ad517-aad5-4175-b444-9a016c517dc5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:00.194869 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ba2ad517-aad5-4175-b444-9a016c517dc5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:00.199543 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ba2ad517-aad5-4175-b444-9a016c517dc5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:00.364114 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-8dc08d29-3597-4f2b-9320-62241464bec7 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Flavor limits 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 19 11:05:00.364594 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-8dc08d29-3597-4f2b-9320-62241464bec7 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Image limits 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 19 11:05:00.365063 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-8dc08d29-3597-4f2b-9320-62241464bec7 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Flavor pref 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 19 11:05:00.366014 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-8dc08d29-3597-4f2b-9320-62241464bec7 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Image pref 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 19 11:05:00.366507 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-8dc08d29-3597-4f2b-9320-62241464bec7 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 19 11:05:00.380918 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8dc08d29-3597-4f2b-9320-62241464bec7 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:00.381443 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8dc08d29-3597-4f2b-9320-62241464bec7 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:00.382236 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8dc08d29-3597-4f2b-9320-62241464bec7 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:00.382686 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8dc08d29-3597-4f2b-9320-62241464bec7 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:00.429275 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-8dc08d29-3597-4f2b-9320-62241464bec7 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Getting quotas for project ef534f1d06ea415d98857ec7f9e5fb97. Resources: {'cores', 'instances', 'ram'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:05:00.437786 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-8dc08d29-3597-4f2b-9320-62241464bec7 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Getting quotas for user b8f2a78f3a3e4b8e8937e3bc39524690 and project ef534f1d06ea415d98857ec7f9e5fb97. Resources: {'cores', 'instances', 'ram'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:05:00.470703 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-8dc08d29-3597-4f2b-9320-62241464bec7 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Going to run 1 instances... {{(pid=117849) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 19 11:05:00.473696 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-8dc08d29-3597-4f2b-9320-62241464bec7 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='3085883e-2289-4314-aa47-9c41d4368b02',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117849) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 19 11:05:00.500127 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-ba2ad517-aad5-4175-b444-9a016c517dc5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:00.509690 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-ba2ad517-aad5-4175-b444-9a016c517dc5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:00.530897 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-ba2ad517-aad5-4175-b444-9a016c517dc5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:00.532531 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-ba2ad517-aad5-4175-b444-9a016c517dc5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1338 microversion: 2.20 time: 0.374523 Jul 19 11:05:00.533772 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 36/73] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:05:00 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1338 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:05:00.544543 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-8ad61046-b885-427a-b3d9-b48e84e6128f tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:00.545619 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-8ad61046-b885-427a-b3d9-b48e84e6128f tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:00.551344 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-8dc08d29-3597-4f2b-9320-62241464bec7 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 3.804498 Jul 19 11:05:00.553948 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8ad61046-b885-427a-b3d9-b48e84e6128f tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:00.554693 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8ad61046-b885-427a-b3d9-b48e84e6128f tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:00.555870 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 38/74] 10.222.0.33 () {60 vars in 1146 bytes} [Tue Jul 19 11:04:56 2022] POST /compute/v2.1/servers => generated 402 bytes in 3810 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 19 11:05:00.556186 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8ad61046-b885-427a-b3d9-b48e84e6128f tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:00.557013 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8ad61046-b885-427a-b3d9-b48e84e6128f tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:00.566550 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-fd310474-dd97-4511-a39e-45719ea1a8a2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:00.567177 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-fd310474-dd97-4511-a39e-45719ea1a8a2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:00.593115 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=100 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:05:00.789761 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-fd310474-dd97-4511-a39e-45719ea1a8a2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:00.798158 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-fd310474-dd97-4511-a39e-45719ea1a8a2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1241 microversion: 2.1 time: 0.236251 Jul 19 11:05:00.800405 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 39/75] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:00 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1241 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:00.807861 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-544b1fe4-db11-4e63-83c5-1e86c1718443 req-53be6684-13a3-4a69-9502-0f996abcdd44 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c3ea5a95-376a-4482-b664-f069e803e258", "tag": "1504f06f-d7c2-447b-acc1-5498633749e3"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:05:00.822711 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-544b1fe4-db11-4e63-83c5-1e86c1718443 req-53be6684-13a3-4a69-9502-0f996abcdd44 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:00.823507 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-544b1fe4-db11-4e63-83c5-1e86c1718443 req-53be6684-13a3-4a69-9502-0f996abcdd44 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:00.905335 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-544b1fe4-db11-4e63-83c5-1e86c1718443 req-53be6684-13a3-4a69-9502-0f996abcdd44 service nova] Creating event network-changed:1504f06f-d7c2-447b-acc1-5498633749e3 for instance c3ea5a95-376a-4482-b664-f069e803e258 on hv-cinder-maste Jul 19 11:05:00.905335 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-544b1fe4-db11-4e63-83c5-1e86c1718443 req-53be6684-13a3-4a69-9502-0f996abcdd44 service nova] Acquired lock "compute-rpcapi-router" {{(pid=117849) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} Jul 19 11:05:00.905648 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2fe40832-ff67-4090-8f6c-2693d7c8be20 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:00.905835 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2fe40832-ff67-4090-8f6c-2693d7c8be20 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:00.907157 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2fe40832-ff67-4090-8f6c-2693d7c8be20 None None] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:00.907729 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2fe40832-ff67-4090-8f6c-2693d7c8be20 None None] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:00.959220 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.compute.rpcapi [None req-2fe40832-ff67-4090-8f6c-2693d7c8be20 None None] Automatically selected compute RPC version 6.0 from minimum service version 61 Jul 19 11:05:00.960530 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2fe40832-ff67-4090-8f6c-2693d7c8be20 None None] Releasing lock "compute-rpcapi-router" {{(pid=117849) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Jul 19 11:05:00.968688 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell0:SELECT=28 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:05:01.007479 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-2fe40832-ff67-4090-8f6c-2693d7c8be20 None None] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.204156 Jul 19 11:05:01.008923 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 40/76] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:05:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:05:01.059843 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-8ad61046-b885-427a-b3d9-b48e84e6128f tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:01.069553 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-8ad61046-b885-427a-b3d9-b48e84e6128f tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1314 microversion: 2.1 time: 0.533901 Jul 19 11:05:01.079081 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 37/77] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:00 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1314 bytes in 544 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:01.172043 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-a95f246f-b86e-4c21-bea0-a3eed5a39bc2 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:01.172800 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-a95f246f-b86e-4c21-bea0-a3eed5a39bc2 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:01.190616 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a95f246f-b86e-4c21-bea0-a3eed5a39bc2 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:01.194609 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a95f246f-b86e-4c21-bea0-a3eed5a39bc2 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:01.198042 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a95f246f-b86e-4c21-bea0-a3eed5a39bc2 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:01.198405 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a95f246f-b86e-4c21-bea0-a3eed5a39bc2 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:01.467208 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-a95f246f-b86e-4c21-bea0-a3eed5a39bc2 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:01.476542 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-a95f246f-b86e-4c21-bea0-a3eed5a39bc2 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 0.308789 Jul 19 11:05:01.483606 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 41/78] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:01 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:01.557360 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-d13484a5-5917-46b6-860e-a6bd84ca5bd8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:01.558158 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-d13484a5-5917-46b6-860e-a6bd84ca5bd8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:01.569058 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d13484a5-5917-46b6-860e-a6bd84ca5bd8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:01.569515 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d13484a5-5917-46b6-860e-a6bd84ca5bd8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:01.570299 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d13484a5-5917-46b6-860e-a6bd84ca5bd8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:01.570732 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d13484a5-5917-46b6-860e-a6bd84ca5bd8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:01.720802 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-d13484a5-5917-46b6-860e-a6bd84ca5bd8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:01.730341 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-d13484a5-5917-46b6-860e-a6bd84ca5bd8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:01.731946 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-d13484a5-5917-46b6-860e-a6bd84ca5bd8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:01.733223 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-d13484a5-5917-46b6-860e-a6bd84ca5bd8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1405 microversion: 2.20 time: 0.183122 Jul 19 11:05:01.733908 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 38/79] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:05:01 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1405 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:05:01.816841 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-0180aed1-dccd-4f82-8205-71e7fd493b1c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:01.817459 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-0180aed1-dccd-4f82-8205-71e7fd493b1c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:01.825359 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0180aed1-dccd-4f82-8205-71e7fd493b1c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:01.826008 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0180aed1-dccd-4f82-8205-71e7fd493b1c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:01.826850 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0180aed1-dccd-4f82-8205-71e7fd493b1c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:01.827228 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0180aed1-dccd-4f82-8205-71e7fd493b1c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:01.911809 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-0180aed1-dccd-4f82-8205-71e7fd493b1c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:01.920320 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-0180aed1-dccd-4f82-8205-71e7fd493b1c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1245 microversion: 2.1 time: 0.107073 Jul 19 11:05:01.932993 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 42/80] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:01 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1245 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:02.087144 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-eb0e6d4f-a380-48ec-b0f1-668bf947c3c9 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:02.088078 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-eb0e6d4f-a380-48ec-b0f1-668bf947c3c9 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:02.094713 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-eb0e6d4f-a380-48ec-b0f1-668bf947c3c9 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:02.095170 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-eb0e6d4f-a380-48ec-b0f1-668bf947c3c9 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:02.096163 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-eb0e6d4f-a380-48ec-b0f1-668bf947c3c9 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:02.096832 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-eb0e6d4f-a380-48ec-b0f1-668bf947c3c9 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:02.254486 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-eb0e6d4f-a380-48ec-b0f1-668bf947c3c9 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:02.264890 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-eb0e6d4f-a380-48ec-b0f1-668bf947c3c9 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1314 microversion: 2.1 time: 0.182343 Jul 19 11:05:02.265661 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 39/81] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:02 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1314 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:02.499844 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-454a599c-6b27-404f-ab7f-1a3d6ee81618 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:02.500311 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-454a599c-6b27-404f-ab7f-1a3d6ee81618 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:02.509313 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-454a599c-6b27-404f-ab7f-1a3d6ee81618 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:02.509991 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-454a599c-6b27-404f-ab7f-1a3d6ee81618 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:02.510973 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-454a599c-6b27-404f-ab7f-1a3d6ee81618 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:02.511649 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-454a599c-6b27-404f-ab7f-1a3d6ee81618 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:02.705334 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-454a599c-6b27-404f-ab7f-1a3d6ee81618 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:02.716073 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-454a599c-6b27-404f-ab7f-1a3d6ee81618 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 0.220411 Jul 19 11:05:02.728224 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 43/82] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:02 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:02.755584 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-cbe688f2-7e86-4854-a1c4-fb9b0d216fbb tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:02.757103 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-cbe688f2-7e86-4854-a1c4-fb9b0d216fbb tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:02.771943 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cbe688f2-7e86-4854-a1c4-fb9b0d216fbb tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:02.772491 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cbe688f2-7e86-4854-a1c4-fb9b0d216fbb tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:02.775810 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cbe688f2-7e86-4854-a1c4-fb9b0d216fbb tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:02.776262 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cbe688f2-7e86-4854-a1c4-fb9b0d216fbb tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:02.940311 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-7c924c81-becc-47ea-9a9a-c47f677c2878 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:02.940956 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-7c924c81-becc-47ea-9a9a-c47f677c2878 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:02.983122 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7c924c81-becc-47ea-9a9a-c47f677c2878 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:02.986278 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7c924c81-becc-47ea-9a9a-c47f677c2878 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.017s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:02.992452 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7c924c81-becc-47ea-9a9a-c47f677c2878 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:02.992948 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7c924c81-becc-47ea-9a9a-c47f677c2878 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:03.223611 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-7c924c81-becc-47ea-9a9a-c47f677c2878 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:03.236144 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-7c924c81-becc-47ea-9a9a-c47f677c2878 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1245 microversion: 2.1 time: 0.300255 Jul 19 11:05:03.236746 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 44/83] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:02 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1245 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:03.283684 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-e83eb5cd-a881-478a-969d-cab2d9cb6b0e tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:03.284640 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-e83eb5cd-a881-478a-969d-cab2d9cb6b0e tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:03.295202 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e83eb5cd-a881-478a-969d-cab2d9cb6b0e tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:03.295667 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e83eb5cd-a881-478a-969d-cab2d9cb6b0e tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:03.296461 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e83eb5cd-a881-478a-969d-cab2d9cb6b0e tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:03.296941 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e83eb5cd-a881-478a-969d-cab2d9cb6b0e tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:03.440160 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-cbe688f2-7e86-4854-a1c4-fb9b0d216fbb tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:03.449532 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-cbe688f2-7e86-4854-a1c4-fb9b0d216fbb tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:03.451438 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-cbe688f2-7e86-4854-a1c4-fb9b0d216fbb tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:03.452835 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-cbe688f2-7e86-4854-a1c4-fb9b0d216fbb tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1405 microversion: 2.20 time: 0.705125 Jul 19 11:05:03.454178 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 40/84] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:05:02 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1405 bytes in 707 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:05:03.738377 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-defe99d6-c8c2-4a0d-86ba-777f9f50be47 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:03.738377 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-defe99d6-c8c2-4a0d-86ba-777f9f50be47 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:03.755268 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-defe99d6-c8c2-4a0d-86ba-777f9f50be47 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:03.755268 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-defe99d6-c8c2-4a0d-86ba-777f9f50be47 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:03.761942 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-defe99d6-c8c2-4a0d-86ba-777f9f50be47 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:03.762568 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-defe99d6-c8c2-4a0d-86ba-777f9f50be47 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:03.912640 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-e83eb5cd-a881-478a-969d-cab2d9cb6b0e tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:03.915002 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=97 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:05:03.932442 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-e83eb5cd-a881-478a-969d-cab2d9cb6b0e tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1314 microversion: 2.1 time: 0.652990 Jul 19 11:05:03.933557 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 45/85] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:03 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1314 bytes in 655 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:04.253408 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-eff009cd-0f8e-4bdd-8926-1bd86a22ba1e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:04.254036 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-eff009cd-0f8e-4bdd-8926-1bd86a22ba1e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:04.259783 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_api:SELECT=84,nova_api:INSERT=7 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:05:04.265445 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-eff009cd-0f8e-4bdd-8926-1bd86a22ba1e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:04.265904 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-eff009cd-0f8e-4bdd-8926-1bd86a22ba1e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:04.266711 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-eff009cd-0f8e-4bdd-8926-1bd86a22ba1e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:04.267153 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-eff009cd-0f8e-4bdd-8926-1bd86a22ba1e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:04.366751 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-eff009cd-0f8e-4bdd-8926-1bd86a22ba1e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:04.377534 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-eff009cd-0f8e-4bdd-8926-1bd86a22ba1e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1299 microversion: 2.1 time: 0.126112 Jul 19 11:05:04.383382 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 46/86] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:04 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1299 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:05:04.474069 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-d2d35c41-b150-4c1d-911b-96dabc798877 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:04.475459 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-d2d35c41-b150-4c1d-911b-96dabc798877 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:04.482922 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d2d35c41-b150-4c1d-911b-96dabc798877 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:04.483563 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d2d35c41-b150-4c1d-911b-96dabc798877 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:04.484334 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d2d35c41-b150-4c1d-911b-96dabc798877 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:04.485246 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d2d35c41-b150-4c1d-911b-96dabc798877 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:04.518954 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-defe99d6-c8c2-4a0d-86ba-777f9f50be47 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:04.532554 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-defe99d6-c8c2-4a0d-86ba-777f9f50be47 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 0.799410 Jul 19 11:05:04.543216 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 41/87] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:03 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 812 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:05:04.694634 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-d2d35c41-b150-4c1d-911b-96dabc798877 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:04.707194 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-d2d35c41-b150-4c1d-911b-96dabc798877 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:04.710550 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-d2d35c41-b150-4c1d-911b-96dabc798877 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:04.712023 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-d2d35c41-b150-4c1d-911b-96dabc798877 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1405 microversion: 2.20 time: 0.243541 Jul 19 11:05:04.712873 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 47/88] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:05:04 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1405 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:05:04.952990 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-79311517-c828-453b-a31e-4a6294c97614 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:04.953849 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-79311517-c828-453b-a31e-4a6294c97614 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:04.962975 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-79311517-c828-453b-a31e-4a6294c97614 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:04.963435 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-79311517-c828-453b-a31e-4a6294c97614 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:04.966415 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-79311517-c828-453b-a31e-4a6294c97614 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:04.966891 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-79311517-c828-453b-a31e-4a6294c97614 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:05.258678 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-79311517-c828-453b-a31e-4a6294c97614 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:05.259487 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-aedd3038-f766-4913-8da5-489684b10588 req-507d4f99-4c6a-4b0e-8e49-66a04a17519f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f52fbfae-577b-403d-a4d3-9a4b79ae2326", "tag": "748a7315-79e9-4ab8-ad6a-336c057e8b2c"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:05:05.269292 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-79311517-c828-453b-a31e-4a6294c97614 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1314 microversion: 2.1 time: 0.322860 Jul 19 11:05:05.269988 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 42/89] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:04 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1314 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:05.290861 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-aedd3038-f766-4913-8da5-489684b10588 req-507d4f99-4c6a-4b0e-8e49-66a04a17519f service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:05.292495 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-aedd3038-f766-4913-8da5-489684b10588 req-507d4f99-4c6a-4b0e-8e49-66a04a17519f service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:05.308669 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-aedd3038-f766-4913-8da5-489684b10588 req-507d4f99-4c6a-4b0e-8e49-66a04a17519f service nova] Creating event network-changed:748a7315-79e9-4ab8-ad6a-336c057e8b2c for instance f52fbfae-577b-403d-a4d3-9a4b79ae2326 on hv-cinder-maste Jul 19 11:05:05.314155 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-aedd3038-f766-4913-8da5-489684b10588 req-507d4f99-4c6a-4b0e-8e49-66a04a17519f service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.061476 Jul 19 11:05:05.315253 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 48/90] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:05:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:05:05.393968 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-93bc5c23-d2c5-47d1-83c3-fdb71abd16c5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:05.394861 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-93bc5c23-d2c5-47d1-83c3-fdb71abd16c5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:05.402506 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-93bc5c23-d2c5-47d1-83c3-fdb71abd16c5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:05.402960 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-93bc5c23-d2c5-47d1-83c3-fdb71abd16c5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:05.405500 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-93bc5c23-d2c5-47d1-83c3-fdb71abd16c5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:05.405952 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-93bc5c23-d2c5-47d1-83c3-fdb71abd16c5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:05.501712 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-93bc5c23-d2c5-47d1-83c3-fdb71abd16c5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:05.526161 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-93bc5c23-d2c5-47d1-83c3-fdb71abd16c5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1299 microversion: 2.1 time: 0.133391 Jul 19 11:05:05.526161 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 43/91] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:05 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1299 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:05.556339 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-b1b0de73-21c2-4a01-8e0a-f05839b9eb10 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:05.560070 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-b1b0de73-21c2-4a01-8e0a-f05839b9eb10 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:05.579921 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b1b0de73-21c2-4a01-8e0a-f05839b9eb10 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:05.580447 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b1b0de73-21c2-4a01-8e0a-f05839b9eb10 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:05.582976 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b1b0de73-21c2-4a01-8e0a-f05839b9eb10 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:05.587111 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b1b0de73-21c2-4a01-8e0a-f05839b9eb10 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:05.732545 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-782a3398-7f0c-4ff7-a31d-fb5aaf61ad27 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:05.733211 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-782a3398-7f0c-4ff7-a31d-fb5aaf61ad27 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:05.743647 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-782a3398-7f0c-4ff7-a31d-fb5aaf61ad27 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:05.744103 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-782a3398-7f0c-4ff7-a31d-fb5aaf61ad27 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:05.745986 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-782a3398-7f0c-4ff7-a31d-fb5aaf61ad27 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:05.746444 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-782a3398-7f0c-4ff7-a31d-fb5aaf61ad27 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:05.812803 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b1b0de73-21c2-4a01-8e0a-f05839b9eb10 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:05.821149 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-b1b0de73-21c2-4a01-8e0a-f05839b9eb10 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 0.274267 Jul 19 11:05:05.821735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 49/92] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:05 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:05.969986 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-782a3398-7f0c-4ff7-a31d-fb5aaf61ad27 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:05.978471 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-782a3398-7f0c-4ff7-a31d-fb5aaf61ad27 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:05.991972 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-782a3398-7f0c-4ff7-a31d-fb5aaf61ad27 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:05.993168 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-782a3398-7f0c-4ff7-a31d-fb5aaf61ad27 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1405 microversion: 2.20 time: 0.265296 Jul 19 11:05:05.997905 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 44/93] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:05:05 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1405 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:05:06.287841 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-c5b3e496-0cf4-4397-9be0-3c48c9feadf6 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:06.288657 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-c5b3e496-0cf4-4397-9be0-3c48c9feadf6 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:06.295798 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c5b3e496-0cf4-4397-9be0-3c48c9feadf6 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:06.296594 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c5b3e496-0cf4-4397-9be0-3c48c9feadf6 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:06.297937 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c5b3e496-0cf4-4397-9be0-3c48c9feadf6 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:06.298478 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c5b3e496-0cf4-4397-9be0-3c48c9feadf6 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:06.540238 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-0f2f2e6d-26cb-4ccc-843e-a034f7c50113 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:06.540958 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-0f2f2e6d-26cb-4ccc-843e-a034f7c50113 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:06.548608 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell0:SELECT=8 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:05:06.550436 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0f2f2e6d-26cb-4ccc-843e-a034f7c50113 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:06.553319 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0f2f2e6d-26cb-4ccc-843e-a034f7c50113 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:06.556987 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0f2f2e6d-26cb-4ccc-843e-a034f7c50113 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:06.557682 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0f2f2e6d-26cb-4ccc-843e-a034f7c50113 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:06.630153 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-c5b3e496-0cf4-4397-9be0-3c48c9feadf6 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:06.642939 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-c5b3e496-0cf4-4397-9be0-3c48c9feadf6 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1314 microversion: 2.1 time: 0.360907 Jul 19 11:05:06.643860 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 50/94] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:06 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1314 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:06.813630 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-0f2f2e6d-26cb-4ccc-843e-a034f7c50113 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:06.827343 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-0f2f2e6d-26cb-4ccc-843e-a034f7c50113 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1299 microversion: 2.1 time: 0.287096 Jul 19 11:05:06.828410 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 45/95] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:06 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1299 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:06.841884 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-a6f14eb4-eca0-4732-854d-d39fe442f714 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:06.842482 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-a6f14eb4-eca0-4732-854d-d39fe442f714 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:06.855751 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a6f14eb4-eca0-4732-854d-d39fe442f714 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:06.856210 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a6f14eb4-eca0-4732-854d-d39fe442f714 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:06.856994 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a6f14eb4-eca0-4732-854d-d39fe442f714 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:06.857569 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a6f14eb4-eca0-4732-854d-d39fe442f714 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:07.015796 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-cbb6214f-44fe-4ff9-a835-153d41d82b4f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:07.016592 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-cbb6214f-44fe-4ff9-a835-153d41d82b4f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:07.027115 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cbb6214f-44fe-4ff9-a835-153d41d82b4f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:07.028112 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cbb6214f-44fe-4ff9-a835-153d41d82b4f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:07.031130 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cbb6214f-44fe-4ff9-a835-153d41d82b4f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:07.031798 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cbb6214f-44fe-4ff9-a835-153d41d82b4f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:07.527671 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-a6f14eb4-eca0-4732-854d-d39fe442f714 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:07.538856 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-a6f14eb4-eca0-4732-854d-d39fe442f714 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 0.701230 Jul 19 11:05:07.541909 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 51/96] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:06 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 705 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:07.578575 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-cbb6214f-44fe-4ff9-a835-153d41d82b4f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:07.589528 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-cbb6214f-44fe-4ff9-a835-153d41d82b4f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:07.591950 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-cbb6214f-44fe-4ff9-a835-153d41d82b4f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:07.593636 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-cbb6214f-44fe-4ff9-a835-153d41d82b4f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1405 microversion: 2.20 time: 0.582063 Jul 19 11:05:07.605163 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 46/97] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:05:07 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1405 bytes in 594 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:05:07.661276 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-a5a0c58c-772a-4181-8d4a-7ce299b02df0 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:07.662033 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-a5a0c58c-772a-4181-8d4a-7ce299b02df0 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:07.672339 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a5a0c58c-772a-4181-8d4a-7ce299b02df0 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:07.674870 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a5a0c58c-772a-4181-8d4a-7ce299b02df0 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:07.677214 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a5a0c58c-772a-4181-8d4a-7ce299b02df0 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:07.687261 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a5a0c58c-772a-4181-8d4a-7ce299b02df0 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:07.844879 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-6a9cb0e0-4652-4fb1-ab53-bf61619e5018 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:07.845595 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-6a9cb0e0-4652-4fb1-ab53-bf61619e5018 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:07.854944 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6a9cb0e0-4652-4fb1-ab53-bf61619e5018 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:07.856527 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6a9cb0e0-4652-4fb1-ab53-bf61619e5018 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:07.861378 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6a9cb0e0-4652-4fb1-ab53-bf61619e5018 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:07.862140 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6a9cb0e0-4652-4fb1-ab53-bf61619e5018 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:07.949965 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-6a9cb0e0-4652-4fb1-ab53-bf61619e5018 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:07.960865 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-6a9cb0e0-4652-4fb1-ab53-bf61619e5018 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1299 microversion: 2.1 time: 0.118493 Jul 19 11:05:07.962368 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 47/98] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:07 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1299 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:05:08.283125 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-a5a0c58c-772a-4181-8d4a-7ce299b02df0 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:08.292091 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-a5a0c58c-772a-4181-8d4a-7ce299b02df0 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1314 microversion: 2.1 time: 0.635169 Jul 19 11:05:08.302038 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 52/99] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:07 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1314 bytes in 646 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:08.558106 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-b72cf501-18b9-43ee-9865-475849a19ebd tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:08.559390 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-b72cf501-18b9-43ee-9865-475849a19ebd tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:08.570365 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b72cf501-18b9-43ee-9865-475849a19ebd tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:08.571162 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b72cf501-18b9-43ee-9865-475849a19ebd tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:08.572151 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b72cf501-18b9-43ee-9865-475849a19ebd tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:08.575070 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b72cf501-18b9-43ee-9865-475849a19ebd tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:08.612371 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-8520c7e2-ab6b-4d4e-8a8b-c405d5eff755 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:08.613568 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-8520c7e2-ab6b-4d4e-8a8b-c405d5eff755 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:08.621066 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8520c7e2-ab6b-4d4e-8a8b-c405d5eff755 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:08.622925 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8520c7e2-ab6b-4d4e-8a8b-c405d5eff755 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:08.628687 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8520c7e2-ab6b-4d4e-8a8b-c405d5eff755 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:08.632748 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8520c7e2-ab6b-4d4e-8a8b-c405d5eff755 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:08.840242 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-8520c7e2-ab6b-4d4e-8a8b-c405d5eff755 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:08.849802 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-8520c7e2-ab6b-4d4e-8a8b-c405d5eff755 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:08.851336 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-8520c7e2-ab6b-4d4e-8a8b-c405d5eff755 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:08.852437 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-8520c7e2-ab6b-4d4e-8a8b-c405d5eff755 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1405 microversion: 2.20 time: 0.244418 Jul 19 11:05:08.853998 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 53/100] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:05:08 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1405 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:05:08.856424 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-b72cf501-18b9-43ee-9865-475849a19ebd tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:08.866868 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-b72cf501-18b9-43ee-9865-475849a19ebd tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 0.311055 Jul 19 11:05:08.869173 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 48/101] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:08 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:08.980261 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-3edf6765-efdc-439a-99e3-a5b6057633c9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:08.981210 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-3edf6765-efdc-439a-99e3-a5b6057633c9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:08.994318 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3edf6765-efdc-439a-99e3-a5b6057633c9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:08.994820 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3edf6765-efdc-439a-99e3-a5b6057633c9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:09.004898 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3edf6765-efdc-439a-99e3-a5b6057633c9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:09.005702 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3edf6765-efdc-439a-99e3-a5b6057633c9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:09.238515 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-3edf6765-efdc-439a-99e3-a5b6057633c9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:09.248539 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-3edf6765-efdc-439a-99e3-a5b6057633c9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1367 microversion: 2.1 time: 0.273117 Jul 19 11:05:09.258815 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 54/102] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:08 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1367 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:09.309287 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-e8bc0127-05a7-41d9-83fc-8a0cf74d39ba tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:09.309903 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-e8bc0127-05a7-41d9-83fc-8a0cf74d39ba tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:09.317397 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e8bc0127-05a7-41d9-83fc-8a0cf74d39ba tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:09.318128 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e8bc0127-05a7-41d9-83fc-8a0cf74d39ba tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:09.319204 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e8bc0127-05a7-41d9-83fc-8a0cf74d39ba tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:09.319799 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e8bc0127-05a7-41d9-83fc-8a0cf74d39ba tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:09.473533 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-e8bc0127-05a7-41d9-83fc-8a0cf74d39ba tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:09.481058 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-e8bc0127-05a7-41d9-83fc-8a0cf74d39ba tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1314 microversion: 2.1 time: 0.175663 Jul 19 11:05:09.481629 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 49/103] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:09 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1314 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:09.870893 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-0afbdf90-b0ec-486a-8e15-a66881a24abc tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:09.871527 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-0afbdf90-b0ec-486a-8e15-a66881a24abc tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:09.877891 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0afbdf90-b0ec-486a-8e15-a66881a24abc tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:09.878543 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0afbdf90-b0ec-486a-8e15-a66881a24abc tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:09.879504 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0afbdf90-b0ec-486a-8e15-a66881a24abc tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:09.879943 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0afbdf90-b0ec-486a-8e15-a66881a24abc tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:09.887826 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-4098871b-8427-4f9a-b37c-185a91024b72 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:09.888498 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-4098871b-8427-4f9a-b37c-185a91024b72 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:09.897389 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4098871b-8427-4f9a-b37c-185a91024b72 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:09.897844 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4098871b-8427-4f9a-b37c-185a91024b72 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:09.899004 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4098871b-8427-4f9a-b37c-185a91024b72 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:09.899441 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4098871b-8427-4f9a-b37c-185a91024b72 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:10.053987 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-0afbdf90-b0ec-486a-8e15-a66881a24abc tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:10.062508 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-0afbdf90-b0ec-486a-8e15-a66881a24abc tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:10.064391 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-0afbdf90-b0ec-486a-8e15-a66881a24abc tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:10.065737 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-0afbdf90-b0ec-486a-8e15-a66881a24abc tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1405 microversion: 2.20 time: 0.198707 Jul 19 11:05:10.066296 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 55/104] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:05:09 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1405 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:05:10.101280 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-4098871b-8427-4f9a-b37c-185a91024b72 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:10.111396 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-4098871b-8427-4f9a-b37c-185a91024b72 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 0.228658 Jul 19 11:05:10.113403 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 50/105] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:09 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:05:10.265142 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-55a26104-8ec5-4daf-81f5-db4f974b4824 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:10.265905 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-55a26104-8ec5-4daf-81f5-db4f974b4824 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:10.272090 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-55a26104-8ec5-4daf-81f5-db4f974b4824 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:10.272545 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-55a26104-8ec5-4daf-81f5-db4f974b4824 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:10.273354 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-55a26104-8ec5-4daf-81f5-db4f974b4824 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:10.273793 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-55a26104-8ec5-4daf-81f5-db4f974b4824 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:10.418920 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-55a26104-8ec5-4daf-81f5-db4f974b4824 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:10.428970 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-55a26104-8ec5-4daf-81f5-db4f974b4824 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1367 microversion: 2.1 time: 0.167548 Jul 19 11:05:10.429747 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 56/106] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:10 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1367 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:10.498078 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-8b792773-977c-41f9-9c87-65b610a24ae9 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:10.499263 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-8b792773-977c-41f9-9c87-65b610a24ae9 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:10.506004 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8b792773-977c-41f9-9c87-65b610a24ae9 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:10.506464 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8b792773-977c-41f9-9c87-65b610a24ae9 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:10.507231 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8b792773-977c-41f9-9c87-65b610a24ae9 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:10.508410 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8b792773-977c-41f9-9c87-65b610a24ae9 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:10.656437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-8b792773-977c-41f9-9c87-65b610a24ae9 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:10.664958 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-8b792773-977c-41f9-9c87-65b610a24ae9 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1314 microversion: 2.1 time: 0.171183 Jul 19 11:05:10.674563 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 51/107] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:10 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1314 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:11.081720 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell0:SELECT=8 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:05:11.083797 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell0:SELECT=4 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:05:11.089664 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-e6c24aed-b80a-4817-a4d4-40593dc8c0b7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:11.090291 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-e6c24aed-b80a-4817-a4d4-40593dc8c0b7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:11.100201 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e6c24aed-b80a-4817-a4d4-40593dc8c0b7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:11.100564 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e6c24aed-b80a-4817-a4d4-40593dc8c0b7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:11.101526 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e6c24aed-b80a-4817-a4d4-40593dc8c0b7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:11.101965 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e6c24aed-b80a-4817-a4d4-40593dc8c0b7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:11.131525 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-62d4b4b1-ea6a-4cf3-a2ed-88a35adbac5e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:11.132150 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-62d4b4b1-ea6a-4cf3-a2ed-88a35adbac5e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:11.146558 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-62d4b4b1-ea6a-4cf3-a2ed-88a35adbac5e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:11.147278 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-62d4b4b1-ea6a-4cf3-a2ed-88a35adbac5e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:11.149439 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-62d4b4b1-ea6a-4cf3-a2ed-88a35adbac5e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:11.150815 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-62d4b4b1-ea6a-4cf3-a2ed-88a35adbac5e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:11.352098 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-62d4b4b1-ea6a-4cf3-a2ed-88a35adbac5e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:11.360631 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-62d4b4b1-ea6a-4cf3-a2ed-88a35adbac5e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 0.234107 Jul 19 11:05:11.368388 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 52/108] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:11 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:11.446033 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-c44e1e69-854e-445c-92ea-a9bf0f5db5cc tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:11.446650 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-c44e1e69-854e-445c-92ea-a9bf0f5db5cc tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:11.454187 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c44e1e69-854e-445c-92ea-a9bf0f5db5cc tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:11.465149 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c44e1e69-854e-445c-92ea-a9bf0f5db5cc tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:11.466669 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c44e1e69-854e-445c-92ea-a9bf0f5db5cc tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:11.467227 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c44e1e69-854e-445c-92ea-a9bf0f5db5cc tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:11.640878 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-c44e1e69-854e-445c-92ea-a9bf0f5db5cc tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:11.655692 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-e6c24aed-b80a-4817-a4d4-40593dc8c0b7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:11.660426 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-c44e1e69-854e-445c-92ea-a9bf0f5db5cc tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1367 microversion: 2.1 time: 0.218340 Jul 19 11:05:11.666637 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 53/109] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:11 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1367 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:05:11.667452 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-e6c24aed-b80a-4817-a4d4-40593dc8c0b7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:11.669315 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-e6c24aed-b80a-4817-a4d4-40593dc8c0b7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:11.670675 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-e6c24aed-b80a-4817-a4d4-40593dc8c0b7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1405 microversion: 2.20 time: 0.590769 Jul 19 11:05:11.672266 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 57/110] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:05:11 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1405 bytes in 593 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:05:11.682312 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-3d4ea32c-247b-4a1e-886d-b1f217df9744 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:11.683038 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-3d4ea32c-247b-4a1e-886d-b1f217df9744 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:11.690309 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3d4ea32c-247b-4a1e-886d-b1f217df9744 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:11.690927 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3d4ea32c-247b-4a1e-886d-b1f217df9744 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:11.691780 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3d4ea32c-247b-4a1e-886d-b1f217df9744 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:11.692441 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3d4ea32c-247b-4a1e-886d-b1f217df9744 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:11.699016 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=100 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:05:11.882476 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-3d4ea32c-247b-4a1e-886d-b1f217df9744 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:11.894659 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-3d4ea32c-247b-4a1e-886d-b1f217df9744 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1314 microversion: 2.1 time: 0.216024 Jul 19 11:05:11.895209 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 54/111] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:11 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1314 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:05:12.019900 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-3b44d4cd-e652-47bb-8cce-281ba48894e6 req-673a47d6-fd80-4810-8744-f24715afa4f1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "126c2860-81b1-42fc-8fbd-78178837f2c2", "name": "network-vif-plugged", "status": "completed", "tag": "ab2b6732-5d45-4b40-8888-44a729c6870f"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:05:12.034945 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-3b44d4cd-e652-47bb-8cce-281ba48894e6 req-673a47d6-fd80-4810-8744-f24715afa4f1 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:12.035398 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-3b44d4cd-e652-47bb-8cce-281ba48894e6 req-673a47d6-fd80-4810-8744-f24715afa4f1 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:12.053595 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-3b44d4cd-e652-47bb-8cce-281ba48894e6 req-673a47d6-fd80-4810-8744-f24715afa4f1 service nova] Creating event network-vif-plugged:ab2b6732-5d45-4b40-8888-44a729c6870f for instance 126c2860-81b1-42fc-8fbd-78178837f2c2 on hv-cinder-maste Jul 19 11:05:12.061309 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-3b44d4cd-e652-47bb-8cce-281ba48894e6 req-673a47d6-fd80-4810-8744-f24715afa4f1 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.107999 Jul 19 11:05:12.065958 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 58/112] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:05:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:05:12.292544 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-f4da0f9c-1c1d-4f9a-ac4e-642dfad2a714 req-1d4dc061-d9be-4213-a722-07637c87d5f4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4c71f564-de63-4fcd-88ce-01ecba3d8372", "tag": "526a30d7-5fcf-4c45-86bf-6ddbc902d48c"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:05:12.304441 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-f4da0f9c-1c1d-4f9a-ac4e-642dfad2a714 req-1d4dc061-d9be-4213-a722-07637c87d5f4 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:12.314384 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-f4da0f9c-1c1d-4f9a-ac4e-642dfad2a714 req-1d4dc061-d9be-4213-a722-07637c87d5f4 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:12.337475 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-f4da0f9c-1c1d-4f9a-ac4e-642dfad2a714 req-1d4dc061-d9be-4213-a722-07637c87d5f4 service nova] Creating event network-changed:526a30d7-5fcf-4c45-86bf-6ddbc902d48c for instance 4c71f564-de63-4fcd-88ce-01ecba3d8372 on hv-cinder-maste Jul 19 11:05:12.350476 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-f4da0f9c-1c1d-4f9a-ac4e-642dfad2a714 req-1d4dc061-d9be-4213-a722-07637c87d5f4 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.062627 Jul 19 11:05:12.353936 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 55/113] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:05:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:05:12.378452 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-5bab988b-d600-4a5b-9af8-6a16c2dde88a tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:12.379439 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-5bab988b-d600-4a5b-9af8-6a16c2dde88a tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:12.389110 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5bab988b-d600-4a5b-9af8-6a16c2dde88a tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:12.390370 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5bab988b-d600-4a5b-9af8-6a16c2dde88a tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:12.391371 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5bab988b-d600-4a5b-9af8-6a16c2dde88a tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:12.392628 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5bab988b-d600-4a5b-9af8-6a16c2dde88a tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:12.613055 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-5bab988b-d600-4a5b-9af8-6a16c2dde88a tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:12.623566 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-5bab988b-d600-4a5b-9af8-6a16c2dde88a tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 0.249480 Jul 19 11:05:12.624650 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 59/114] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:12 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:12.688137 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-debdeacf-95da-4c1b-b0b4-bb13e275ff66 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:12.688898 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-28c69dc1-9b30-41d6-90e8-404dac59f76d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:12.689025 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-debdeacf-95da-4c1b-b0b4-bb13e275ff66 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:12.689516 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-28c69dc1-9b30-41d6-90e8-404dac59f76d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:12.695922 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-28c69dc1-9b30-41d6-90e8-404dac59f76d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:12.696377 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-28c69dc1-9b30-41d6-90e8-404dac59f76d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:12.696807 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-debdeacf-95da-4c1b-b0b4-bb13e275ff66 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:12.697185 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-28c69dc1-9b30-41d6-90e8-404dac59f76d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:12.697308 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-debdeacf-95da-4c1b-b0b4-bb13e275ff66 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:12.697628 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-28c69dc1-9b30-41d6-90e8-404dac59f76d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:12.698018 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-debdeacf-95da-4c1b-b0b4-bb13e275ff66 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:12.698453 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-debdeacf-95da-4c1b-b0b4-bb13e275ff66 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:12.899532 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-28c69dc1-9b30-41d6-90e8-404dac59f76d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:12.907824 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-debdeacf-95da-4c1b-b0b4-bb13e275ff66 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:12.909921 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-28c69dc1-9b30-41d6-90e8-404dac59f76d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:12.911451 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-28c69dc1-9b30-41d6-90e8-404dac59f76d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:12.912624 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-28c69dc1-9b30-41d6-90e8-404dac59f76d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1405 microversion: 2.20 time: 0.227736 Jul 19 11:05:12.913214 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 60/115] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:05:12 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1405 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:05:12.916536 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-debdeacf-95da-4c1b-b0b4-bb13e275ff66 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1367 microversion: 2.1 time: 0.235771 Jul 19 11:05:12.918533 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-052c8bc2-035c-4e66-b6f6-32e02b6462b8 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:12.919378 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-052c8bc2-035c-4e66-b6f6-32e02b6462b8 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:12.925755 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 56/116] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:12 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1367 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:12.928498 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-052c8bc2-035c-4e66-b6f6-32e02b6462b8 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:12.928987 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-052c8bc2-035c-4e66-b6f6-32e02b6462b8 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:12.929770 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-052c8bc2-035c-4e66-b6f6-32e02b6462b8 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:12.930214 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-052c8bc2-035c-4e66-b6f6-32e02b6462b8 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:13.074541 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-052c8bc2-035c-4e66-b6f6-32e02b6462b8 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:13.089166 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-052c8bc2-035c-4e66-b6f6-32e02b6462b8 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1314 microversion: 2.1 time: 0.174711 Jul 19 11:05:13.090522 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 61/117] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:12 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1314 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:13.640716 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-488f3908-3b51-4290-a9c1-4e08ac1b8b20 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:13.641317 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-488f3908-3b51-4290-a9c1-4e08ac1b8b20 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:13.647338 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-488f3908-3b51-4290-a9c1-4e08ac1b8b20 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:13.647780 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-488f3908-3b51-4290-a9c1-4e08ac1b8b20 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:13.648530 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-488f3908-3b51-4290-a9c1-4e08ac1b8b20 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:13.649044 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-488f3908-3b51-4290-a9c1-4e08ac1b8b20 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:13.774999 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-488f3908-3b51-4290-a9c1-4e08ac1b8b20 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:13.782130 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-488f3908-3b51-4290-a9c1-4e08ac1b8b20 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 0.145230 Jul 19 11:05:13.782690 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 57/118] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:13 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:13.931264 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-d36705b2-c0c8-4eba-bcb8-0b9c40b3c892 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:13.931849 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-d36705b2-c0c8-4eba-bcb8-0b9c40b3c892 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:13.934172 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-a838f8ac-8982-4a5c-8a57-14fabda6ec43 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:13.935018 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-a838f8ac-8982-4a5c-8a57-14fabda6ec43 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:13.942099 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a838f8ac-8982-4a5c-8a57-14fabda6ec43 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:13.942749 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a838f8ac-8982-4a5c-8a57-14fabda6ec43 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:13.943710 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a838f8ac-8982-4a5c-8a57-14fabda6ec43 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:13.944091 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d36705b2-c0c8-4eba-bcb8-0b9c40b3c892 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:13.944188 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a838f8ac-8982-4a5c-8a57-14fabda6ec43 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:13.944547 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d36705b2-c0c8-4eba-bcb8-0b9c40b3c892 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:13.945352 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d36705b2-c0c8-4eba-bcb8-0b9c40b3c892 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:13.945791 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d36705b2-c0c8-4eba-bcb8-0b9c40b3c892 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:14.085212 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-d36705b2-c0c8-4eba-bcb8-0b9c40b3c892 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:14.087210 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-a838f8ac-8982-4a5c-8a57-14fabda6ec43 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:14.093701 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-d36705b2-c0c8-4eba-bcb8-0b9c40b3c892 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:14.094770 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-a838f8ac-8982-4a5c-8a57-14fabda6ec43 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1367 microversion: 2.1 time: 0.165672 Jul 19 11:05:14.095255 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-d36705b2-c0c8-4eba-bcb8-0b9c40b3c892 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:14.096541 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-d36705b2-c0c8-4eba-bcb8-0b9c40b3c892 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1405 microversion: 2.20 time: 0.169826 Jul 19 11:05:14.097208 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 62/119] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:05:13 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1405 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:05:14.104739 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 58/120] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:13 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1367 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:14.109213 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-652ec811-69ed-4410-8723-c988ac61776d tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:14.109824 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-652ec811-69ed-4410-8723-c988ac61776d tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:14.115728 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-652ec811-69ed-4410-8723-c988ac61776d tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:14.116376 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-652ec811-69ed-4410-8723-c988ac61776d tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:14.117188 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-652ec811-69ed-4410-8723-c988ac61776d tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:14.118255 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-652ec811-69ed-4410-8723-c988ac61776d tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:14.257804 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-652ec811-69ed-4410-8723-c988ac61776d tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:14.260955 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=100 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:05:14.267298 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-652ec811-69ed-4410-8723-c988ac61776d tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1314 microversion: 2.1 time: 0.161712 Jul 19 11:05:14.267866 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 63/121] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:14 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1314 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:05:14.798987 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-e802df26-df3d-48e3-9c96-9bd2d032d0be tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:14.799501 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-e802df26-df3d-48e3-9c96-9bd2d032d0be tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:14.806678 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e802df26-df3d-48e3-9c96-9bd2d032d0be tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:14.806954 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e802df26-df3d-48e3-9c96-9bd2d032d0be tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:14.807830 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e802df26-df3d-48e3-9c96-9bd2d032d0be tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:14.808181 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e802df26-df3d-48e3-9c96-9bd2d032d0be tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:14.981167 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-e802df26-df3d-48e3-9c96-9bd2d032d0be tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:14.988621 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-e802df26-df3d-48e3-9c96-9bd2d032d0be tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 0.193666 Jul 19 11:05:14.993730 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 59/122] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:14 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:15.111434 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-11673e2b-e8b9-40e8-ae24-8ad02bf37bf2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:15.112088 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-11673e2b-e8b9-40e8-ae24-8ad02bf37bf2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:15.116678 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-d546ecb9-e56c-4043-bf5b-8134a769a6af tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:15.117311 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-d546ecb9-e56c-4043-bf5b-8134a769a6af tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:15.118820 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-11673e2b-e8b9-40e8-ae24-8ad02bf37bf2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:15.119308 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-11673e2b-e8b9-40e8-ae24-8ad02bf37bf2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:15.120037 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-11673e2b-e8b9-40e8-ae24-8ad02bf37bf2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:15.120468 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-11673e2b-e8b9-40e8-ae24-8ad02bf37bf2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:15.124753 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d546ecb9-e56c-4043-bf5b-8134a769a6af tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:15.125701 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d546ecb9-e56c-4043-bf5b-8134a769a6af tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:15.126715 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d546ecb9-e56c-4043-bf5b-8134a769a6af tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:15.127216 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d546ecb9-e56c-4043-bf5b-8134a769a6af tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:15.269380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-11673e2b-e8b9-40e8-ae24-8ad02bf37bf2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:15.276813 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-d546ecb9-e56c-4043-bf5b-8134a769a6af tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:15.277653 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-11673e2b-e8b9-40e8-ae24-8ad02bf37bf2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1367 microversion: 2.1 time: 0.170171 Jul 19 11:05:15.278240 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 64/123] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:15 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1367 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:15.284506 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-7f631ee6-0fb0-417d-a116-08256906d1b8 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:15.285154 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-7f631ee6-0fb0-417d-a116-08256906d1b8 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:15.285563 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-d546ecb9-e56c-4043-bf5b-8134a769a6af tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:15.287082 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-d546ecb9-e56c-4043-bf5b-8134a769a6af tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:15.288172 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-d546ecb9-e56c-4043-bf5b-8134a769a6af tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1405 microversion: 2.20 time: 0.176911 Jul 19 11:05:15.288771 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 60/124] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:05:15 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1405 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:05:15.296386 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7f631ee6-0fb0-417d-a116-08256906d1b8 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:15.296877 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7f631ee6-0fb0-417d-a116-08256906d1b8 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:15.297659 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7f631ee6-0fb0-417d-a116-08256906d1b8 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:15.298263 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7f631ee6-0fb0-417d-a116-08256906d1b8 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:15.437773 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-7f631ee6-0fb0-417d-a116-08256906d1b8 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:15.446406 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-7f631ee6-0fb0-417d-a116-08256906d1b8 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1314 microversion: 2.1 time: 0.165038 Jul 19 11:05:15.447122 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 65/125] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:15 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1314 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:05:16.004556 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-825c9783-5c45-4b1a-8528-e0aa6116762a tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:16.005312 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-825c9783-5c45-4b1a-8528-e0aa6116762a tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:16.011719 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-825c9783-5c45-4b1a-8528-e0aa6116762a tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:16.012171 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-825c9783-5c45-4b1a-8528-e0aa6116762a tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:16.012973 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-825c9783-5c45-4b1a-8528-e0aa6116762a tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:16.013414 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-825c9783-5c45-4b1a-8528-e0aa6116762a tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:16.142685 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-825c9783-5c45-4b1a-8528-e0aa6116762a tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:16.150783 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-825c9783-5c45-4b1a-8528-e0aa6116762a tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 0.149781 Jul 19 11:05:16.151522 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 61/126] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:15 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:16.297154 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-d9f9abc3-b721-450d-a213-194b68beb20a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:16.297771 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-d9f9abc3-b721-450d-a213-194b68beb20a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:16.308373 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-dbdafbe6-088d-470d-bcef-5ccffff28f21 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:16.309096 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-dbdafbe6-088d-470d-bcef-5ccffff28f21 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:16.317565 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d9f9abc3-b721-450d-a213-194b68beb20a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:16.318024 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d9f9abc3-b721-450d-a213-194b68beb20a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:16.318796 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d9f9abc3-b721-450d-a213-194b68beb20a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:16.320492 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dbdafbe6-088d-470d-bcef-5ccffff28f21 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:16.320968 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dbdafbe6-088d-470d-bcef-5ccffff28f21 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:16.321787 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dbdafbe6-088d-470d-bcef-5ccffff28f21 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:16.322220 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dbdafbe6-088d-470d-bcef-5ccffff28f21 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:16.326704 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d9f9abc3-b721-450d-a213-194b68beb20a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:16.482131 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-d9f9abc3-b721-450d-a213-194b68beb20a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:16.489632 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-d9f9abc3-b721-450d-a213-194b68beb20a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1367 microversion: 2.1 time: 0.197610 Jul 19 11:05:16.490561 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 66/127] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:16 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1367 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:16.495109 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-c91e4e58-fdf7-49ac-907a-2c1c8771a5e7 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:16.495839 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-c91e4e58-fdf7-49ac-907a-2c1c8771a5e7 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:16.502633 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c91e4e58-fdf7-49ac-907a-2c1c8771a5e7 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:16.503063 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c91e4e58-fdf7-49ac-907a-2c1c8771a5e7 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:16.503821 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c91e4e58-fdf7-49ac-907a-2c1c8771a5e7 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:16.504257 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c91e4e58-fdf7-49ac-907a-2c1c8771a5e7 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:16.505374 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-dbdafbe6-088d-470d-bcef-5ccffff28f21 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:16.513698 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-dbdafbe6-088d-470d-bcef-5ccffff28f21 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:16.515339 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-dbdafbe6-088d-470d-bcef-5ccffff28f21 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:16.516572 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-dbdafbe6-088d-470d-bcef-5ccffff28f21 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1405 microversion: 2.20 time: 0.213863 Jul 19 11:05:16.517168 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 62/128] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:05:16 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1405 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:05:16.637546 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-c91e4e58-fdf7-49ac-907a-2c1c8771a5e7 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:16.646318 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-c91e4e58-fdf7-49ac-907a-2c1c8771a5e7 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1314 microversion: 2.1 time: 0.154688 Jul 19 11:05:16.652969 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 67/129] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:16 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1314 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:17.167953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-a18f7ddc-af7e-4f93-b4a0-50998103ff46 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:17.168663 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-a18f7ddc-af7e-4f93-b4a0-50998103ff46 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:17.174772 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a18f7ddc-af7e-4f93-b4a0-50998103ff46 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:17.175219 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a18f7ddc-af7e-4f93-b4a0-50998103ff46 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:17.175972 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a18f7ddc-af7e-4f93-b4a0-50998103ff46 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:17.176401 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a18f7ddc-af7e-4f93-b4a0-50998103ff46 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:17.347741 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-a18f7ddc-af7e-4f93-b4a0-50998103ff46 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:17.356910 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-a18f7ddc-af7e-4f93-b4a0-50998103ff46 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 0.193293 Jul 19 11:05:17.357984 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 63/130] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:17 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:17.506580 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-dc0e3be0-0b7d-4998-989f-031674f0cfca tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:17.507864 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-dc0e3be0-0b7d-4998-989f-031674f0cfca tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:17.514837 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dc0e3be0-0b7d-4998-989f-031674f0cfca tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:17.515347 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dc0e3be0-0b7d-4998-989f-031674f0cfca tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:17.516278 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dc0e3be0-0b7d-4998-989f-031674f0cfca tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:17.516742 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dc0e3be0-0b7d-4998-989f-031674f0cfca tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:17.536688 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-21119cd4-fe9f-4262-a59b-6038c0e55894 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:17.537742 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-21119cd4-fe9f-4262-a59b-6038c0e55894 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:17.548426 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-21119cd4-fe9f-4262-a59b-6038c0e55894 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:17.548928 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-21119cd4-fe9f-4262-a59b-6038c0e55894 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:17.550607 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-21119cd4-fe9f-4262-a59b-6038c0e55894 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:17.551184 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-21119cd4-fe9f-4262-a59b-6038c0e55894 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:17.700540 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-dc0e3be0-0b7d-4998-989f-031674f0cfca tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:17.716391 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-dc0e3be0-0b7d-4998-989f-031674f0cfca tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1367 microversion: 2.1 time: 0.213460 Jul 19 11:05:17.728913 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 68/131] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:17 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1367 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:17.736653 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-476cbb17-61c7-48e7-8097-1e69591ffc5b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:17.737790 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-476cbb17-61c7-48e7-8097-1e69591ffc5b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:17.749722 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-476cbb17-61c7-48e7-8097-1e69591ffc5b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:17.750475 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-476cbb17-61c7-48e7-8097-1e69591ffc5b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:17.751773 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-476cbb17-61c7-48e7-8097-1e69591ffc5b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:17.752397 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-476cbb17-61c7-48e7-8097-1e69591ffc5b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:17.779185 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-21119cd4-fe9f-4262-a59b-6038c0e55894 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:17.794248 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-21119cd4-fe9f-4262-a59b-6038c0e55894 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:17.798219 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-21119cd4-fe9f-4262-a59b-6038c0e55894 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:17.801557 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-21119cd4-fe9f-4262-a59b-6038c0e55894 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1405 microversion: 2.20 time: 0.270183 Jul 19 11:05:17.802761 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 64/132] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:05:17 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1405 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:05:17.972521 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-476cbb17-61c7-48e7-8097-1e69591ffc5b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:17.984859 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-476cbb17-61c7-48e7-8097-1e69591ffc5b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1314 microversion: 2.1 time: 0.254556 Jul 19 11:05:17.986255 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 69/133] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:17 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1314 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:18.375932 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-79ce6bbe-5251-4311-bf45-08a37c205f5b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:18.377332 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-79ce6bbe-5251-4311-bf45-08a37c205f5b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:18.387132 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-79ce6bbe-5251-4311-bf45-08a37c205f5b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:18.387848 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-79ce6bbe-5251-4311-bf45-08a37c205f5b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:18.388923 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-79ce6bbe-5251-4311-bf45-08a37c205f5b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:18.400377 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-79ce6bbe-5251-4311-bf45-08a37c205f5b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:18.617960 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-79ce6bbe-5251-4311-bf45-08a37c205f5b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:18.629003 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-79ce6bbe-5251-4311-bf45-08a37c205f5b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 0.257458 Jul 19 11:05:18.629788 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 65/134] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:18 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:18.652967 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-67ad98a0-1ee3-4c8b-bc1b-8d32a8187e6b tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-302529431", "imageRef": "3085883e-2289-4314-aa47-9c41d4368b02", "flavorRef": "452", "networks": [{"uuid": "392cf9f9-e9a5-4cb6-9c0a-fce64784a4d2"}]}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:05:18.716447 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.network.neutron [None req-67ad98a0-1ee3-4c8b-bc1b-8d32a8187e6b tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] validate_networks() for [('392cf9f9-e9a5-4cb6-9c0a-fce64784a4d2', None, None, None, None, None)] {{(pid=117849) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 19 11:05:18.737746 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-cd949e0f-289f-4ddf-8f99-3ccb4a4b93b7 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:18.738403 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-cd949e0f-289f-4ddf-8f99-3ccb4a4b93b7 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:18.745249 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cd949e0f-289f-4ddf-8f99-3ccb4a4b93b7 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:18.745789 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cd949e0f-289f-4ddf-8f99-3ccb4a4b93b7 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:18.746720 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cd949e0f-289f-4ddf-8f99-3ccb4a4b93b7 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:18.747830 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cd949e0f-289f-4ddf-8f99-3ccb4a4b93b7 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:18.889083 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-cd949e0f-289f-4ddf-8f99-3ccb4a4b93b7 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:18.897820 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-cd949e0f-289f-4ddf-8f99-3ccb4a4b93b7 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1367 microversion: 2.1 time: 0.163937 Jul 19 11:05:18.899006 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 66/135] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:18 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1367 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:05:18.904765 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-fae5e031-1f70-4f7b-b860-2757a1c203b8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:18.905670 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-fae5e031-1f70-4f7b-b860-2757a1c203b8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:18.912556 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fae5e031-1f70-4f7b-b860-2757a1c203b8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:18.913158 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fae5e031-1f70-4f7b-b860-2757a1c203b8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:18.914032 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fae5e031-1f70-4f7b-b860-2757a1c203b8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:18.915099 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fae5e031-1f70-4f7b-b860-2757a1c203b8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:19.063920 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-fae5e031-1f70-4f7b-b860-2757a1c203b8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:19.078747 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-fae5e031-1f70-4f7b-b860-2757a1c203b8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:19.081462 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-fae5e031-1f70-4f7b-b860-2757a1c203b8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:19.082824 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-fae5e031-1f70-4f7b-b860-2757a1c203b8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1405 microversion: 2.20 time: 0.182460 Jul 19 11:05:19.083832 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 67/136] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:05:18 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1405 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:05:19.089042 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-470040b1-c90f-400c-9b78-bef362e8464a tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:19.090003 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-470040b1-c90f-400c-9b78-bef362e8464a tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:19.097791 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-470040b1-c90f-400c-9b78-bef362e8464a tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:19.098244 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-470040b1-c90f-400c-9b78-bef362e8464a tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:19.099291 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-470040b1-c90f-400c-9b78-bef362e8464a tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:19.099921 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-470040b1-c90f-400c-9b78-bef362e8464a tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:19.319812 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-470040b1-c90f-400c-9b78-bef362e8464a tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:19.327922 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-470040b1-c90f-400c-9b78-bef362e8464a tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1314 microversion: 2.1 time: 0.242926 Jul 19 11:05:19.332840 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 68/137] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:19 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1314 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:19.510588 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-67ad98a0-1ee3-4c8b-bc1b-8d32a8187e6b tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Flavor limits 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 19 11:05:19.510935 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-67ad98a0-1ee3-4c8b-bc1b-8d32a8187e6b tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Image limits 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 19 11:05:19.511441 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-67ad98a0-1ee3-4c8b-bc1b-8d32a8187e6b tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Flavor pref 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 19 11:05:19.512030 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-67ad98a0-1ee3-4c8b-bc1b-8d32a8187e6b tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Image pref 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 19 11:05:19.512452 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-67ad98a0-1ee3-4c8b-bc1b-8d32a8187e6b tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 19 11:05:19.521842 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-67ad98a0-1ee3-4c8b-bc1b-8d32a8187e6b tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:19.522291 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-67ad98a0-1ee3-4c8b-bc1b-8d32a8187e6b tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:19.523089 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-67ad98a0-1ee3-4c8b-bc1b-8d32a8187e6b tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:19.524052 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-67ad98a0-1ee3-4c8b-bc1b-8d32a8187e6b tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:19.541800 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-67ad98a0-1ee3-4c8b-bc1b-8d32a8187e6b tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Getting quotas for project 88dc732812fc4fdd8e9c307f3565ba58. Resources: {'cores', 'instances', 'ram'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:05:19.545831 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-67ad98a0-1ee3-4c8b-bc1b-8d32a8187e6b tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Getting quotas for user e2293054dd2740e781825cbd068d74f7 and project 88dc732812fc4fdd8e9c307f3565ba58. Resources: {'cores', 'instances', 'ram'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:05:19.557297 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-67ad98a0-1ee3-4c8b-bc1b-8d32a8187e6b tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Going to run 1 instances... {{(pid=117849) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 19 11:05:19.559559 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-67ad98a0-1ee3-4c8b-bc1b-8d32a8187e6b tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='3085883e-2289-4314-aa47-9c41d4368b02',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117849) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 19 11:05:19.605213 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-67ad98a0-1ee3-4c8b-bc1b-8d32a8187e6b tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.959661 Jul 19 11:05:19.605918 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 70/138] 10.222.0.33 () {60 vars in 1146 bytes} [Tue Jul 19 11:05:18 2022] POST /compute/v2.1/servers => generated 380 bytes in 961 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 19 11:05:19.616489 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-bd17122f-55ef-4e4a-8e56-a54d22cf8490 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:19.617110 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-bd17122f-55ef-4e4a-8e56-a54d22cf8490 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:19.645772 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-5ed7997f-cc39-409e-b0ca-84c909b7f34e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:19.646876 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-5ed7997f-cc39-409e-b0ca-84c909b7f34e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:19.654980 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5ed7997f-cc39-409e-b0ca-84c909b7f34e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:19.656049 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5ed7997f-cc39-409e-b0ca-84c909b7f34e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:19.656994 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5ed7997f-cc39-409e-b0ca-84c909b7f34e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:19.657566 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5ed7997f-cc39-409e-b0ca-84c909b7f34e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:19.682194 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-bd17122f-55ef-4e4a-8e56-a54d22cf8490 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:19.690254 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-bd17122f-55ef-4e4a-8e56-a54d22cf8490 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1213 microversion: 2.1 time: 0.077632 Jul 19 11:05:19.691299 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 69/139] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:19 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1213 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:19.843593 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-5ed7997f-cc39-409e-b0ca-84c909b7f34e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:19.852684 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-5ed7997f-cc39-409e-b0ca-84c909b7f34e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 0.211053 Jul 19 11:05:19.862965 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 71/140] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:19 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:05:19.915567 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-85db37d5-f5f0-4879-aea4-b7cc754fcafc tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:19.916201 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-85db37d5-f5f0-4879-aea4-b7cc754fcafc tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:19.922735 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-85db37d5-f5f0-4879-aea4-b7cc754fcafc tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:19.923187 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-85db37d5-f5f0-4879-aea4-b7cc754fcafc tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:19.923958 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-85db37d5-f5f0-4879-aea4-b7cc754fcafc tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:19.924394 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-85db37d5-f5f0-4879-aea4-b7cc754fcafc tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:20.061596 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-85db37d5-f5f0-4879-aea4-b7cc754fcafc tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:20.069047 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-85db37d5-f5f0-4879-aea4-b7cc754fcafc tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1367 microversion: 2.1 time: 0.157370 Jul 19 11:05:20.069602 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 70/141] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:19 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1367 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:20.102271 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-8dc9c693-8591-486f-a9f2-87ccb64efefe tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:20.102946 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-8dc9c693-8591-486f-a9f2-87ccb64efefe tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:20.108970 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8dc9c693-8591-486f-a9f2-87ccb64efefe tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:20.109417 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8dc9c693-8591-486f-a9f2-87ccb64efefe tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:20.110163 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8dc9c693-8591-486f-a9f2-87ccb64efefe tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:20.110593 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8dc9c693-8591-486f-a9f2-87ccb64efefe tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:20.235407 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-8dc9c693-8591-486f-a9f2-87ccb64efefe tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:20.243136 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-8dc9c693-8591-486f-a9f2-87ccb64efefe tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:20.244636 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-8dc9c693-8591-486f-a9f2-87ccb64efefe tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:20.245757 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-8dc9c693-8591-486f-a9f2-87ccb64efefe tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1405 microversion: 2.20 time: 0.147842 Jul 19 11:05:20.246315 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 72/142] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:05:20 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1405 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:05:20.344351 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-d322e3e5-6cad-4daf-82dc-e2b07a7edcc3 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:20.344946 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-d322e3e5-6cad-4daf-82dc-e2b07a7edcc3 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:20.350999 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d322e3e5-6cad-4daf-82dc-e2b07a7edcc3 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:20.351446 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d322e3e5-6cad-4daf-82dc-e2b07a7edcc3 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:20.352242 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d322e3e5-6cad-4daf-82dc-e2b07a7edcc3 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:20.352676 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d322e3e5-6cad-4daf-82dc-e2b07a7edcc3 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:20.483953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-d322e3e5-6cad-4daf-82dc-e2b07a7edcc3 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:20.491912 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-d322e3e5-6cad-4daf-82dc-e2b07a7edcc3 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1314 microversion: 2.1 time: 0.151367 Jul 19 11:05:20.492474 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 71/143] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:20 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1314 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:20.707339 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-a2cf9a19-d90e-4d4d-a087-016be4f4c822 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:20.708149 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-a2cf9a19-d90e-4d4d-a087-016be4f4c822 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:20.725136 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a2cf9a19-d90e-4d4d-a087-016be4f4c822 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:20.725584 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a2cf9a19-d90e-4d4d-a087-016be4f4c822 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:20.726350 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a2cf9a19-d90e-4d4d-a087-016be4f4c822 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:20.726803 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a2cf9a19-d90e-4d4d-a087-016be4f4c822 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:20.788620 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-a2cf9a19-d90e-4d4d-a087-016be4f4c822 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:20.796231 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-a2cf9a19-d90e-4d4d-a087-016be4f4c822 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1217 microversion: 2.1 time: 0.093840 Jul 19 11:05:20.796832 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 73/144] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:20 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1217 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:20.818826 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-6493b2d0-70d2-4d0a-9a79-c771123489dd req-0efb2ddb-493c-4c65-808a-0aea33be4046 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c3ea5a95-376a-4482-b664-f069e803e258", "name": "network-vif-plugged", "status": "completed", "tag": "1504f06f-d7c2-447b-acc1-5498633749e3"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:05:20.827613 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-6493b2d0-70d2-4d0a-9a79-c771123489dd req-0efb2ddb-493c-4c65-808a-0aea33be4046 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:20.828204 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-6493b2d0-70d2-4d0a-9a79-c771123489dd req-0efb2ddb-493c-4c65-808a-0aea33be4046 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:20.840303 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-6493b2d0-70d2-4d0a-9a79-c771123489dd req-0efb2ddb-493c-4c65-808a-0aea33be4046 service nova] Creating event network-vif-plugged:1504f06f-d7c2-447b-acc1-5498633749e3 for instance c3ea5a95-376a-4482-b664-f069e803e258 on hv-cinder-maste Jul 19 11:05:20.845451 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-6493b2d0-70d2-4d0a-9a79-c771123489dd req-0efb2ddb-493c-4c65-808a-0aea33be4046 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030233 Jul 19 11:05:20.846122 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 72/145] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:05:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:05:20.870305 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-ba5b3944-655c-489a-b8e7-e14c3968f49d tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:20.870921 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-ba5b3944-655c-489a-b8e7-e14c3968f49d tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:20.877599 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ba5b3944-655c-489a-b8e7-e14c3968f49d tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:20.878047 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ba5b3944-655c-489a-b8e7-e14c3968f49d tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:20.878822 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ba5b3944-655c-489a-b8e7-e14c3968f49d tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:20.879253 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ba5b3944-655c-489a-b8e7-e14c3968f49d tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:21.015806 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-ba5b3944-655c-489a-b8e7-e14c3968f49d tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:21.023549 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-ba5b3944-655c-489a-b8e7-e14c3968f49d tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 0.157468 Jul 19 11:05:21.024177 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 74/146] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:20 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:21.086355 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-54ea5539-9719-4202-97eb-517cb42f856e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:21.087085 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-54ea5539-9719-4202-97eb-517cb42f856e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:21.093324 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-54ea5539-9719-4202-97eb-517cb42f856e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:21.093773 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-54ea5539-9719-4202-97eb-517cb42f856e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:21.094537 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-54ea5539-9719-4202-97eb-517cb42f856e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:21.094971 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-54ea5539-9719-4202-97eb-517cb42f856e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:21.099082 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=100 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:05:21.226286 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-54ea5539-9719-4202-97eb-517cb42f856e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:21.233601 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-54ea5539-9719-4202-97eb-517cb42f856e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1367 microversion: 2.1 time: 0.151532 Jul 19 11:05:21.234172 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 73/147] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:21 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1367 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:21.264156 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-9b2ba0cc-a54e-484a-8c62-4c0061c415c0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:21.264790 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-9b2ba0cc-a54e-484a-8c62-4c0061c415c0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:21.270851 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9b2ba0cc-a54e-484a-8c62-4c0061c415c0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:21.271324 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9b2ba0cc-a54e-484a-8c62-4c0061c415c0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:21.272086 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9b2ba0cc-a54e-484a-8c62-4c0061c415c0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:21.272521 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9b2ba0cc-a54e-484a-8c62-4c0061c415c0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:21.397570 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-9b2ba0cc-a54e-484a-8c62-4c0061c415c0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:21.405316 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-9b2ba0cc-a54e-484a-8c62-4c0061c415c0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:21.407144 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-9b2ba0cc-a54e-484a-8c62-4c0061c415c0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:21.408231 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-9b2ba0cc-a54e-484a-8c62-4c0061c415c0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1405 microversion: 2.20 time: 0.148049 Jul 19 11:05:21.408813 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 75/148] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:05:21 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1405 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:05:21.508927 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-3283ffc7-799c-4708-a5d3-d4c533fc032b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:21.509523 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-3283ffc7-799c-4708-a5d3-d4c533fc032b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:21.515456 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3283ffc7-799c-4708-a5d3-d4c533fc032b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:21.515894 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3283ffc7-799c-4708-a5d3-d4c533fc032b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:21.516636 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3283ffc7-799c-4708-a5d3-d4c533fc032b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:21.517090 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3283ffc7-799c-4708-a5d3-d4c533fc032b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:21.645744 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-3283ffc7-799c-4708-a5d3-d4c533fc032b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:21.652994 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-3283ffc7-799c-4708-a5d3-d4c533fc032b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1314 microversion: 2.1 time: 0.148101 Jul 19 11:05:21.653551 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 74/149] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:21 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1314 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:21.812613 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-9b627a9b-7e3e-40c5-9447-329f36bfee95 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:21.813299 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-9b627a9b-7e3e-40c5-9447-329f36bfee95 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:21.819328 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9b627a9b-7e3e-40c5-9447-329f36bfee95 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:21.819775 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9b627a9b-7e3e-40c5-9447-329f36bfee95 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:21.820523 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9b627a9b-7e3e-40c5-9447-329f36bfee95 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:21.820997 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9b627a9b-7e3e-40c5-9447-329f36bfee95 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:21.880683 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-9b627a9b-7e3e-40c5-9447-329f36bfee95 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:21.888171 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-9b627a9b-7e3e-40c5-9447-329f36bfee95 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1217 microversion: 2.1 time: 0.079345 Jul 19 11:05:21.888757 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 76/150] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:21 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1217 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:22.041457 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-c91adb0c-eeec-4813-a9ae-bb448d6e25cf tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:22.041993 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-c91adb0c-eeec-4813-a9ae-bb448d6e25cf tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:22.048061 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c91adb0c-eeec-4813-a9ae-bb448d6e25cf tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:22.048501 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c91adb0c-eeec-4813-a9ae-bb448d6e25cf tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:22.049289 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c91adb0c-eeec-4813-a9ae-bb448d6e25cf tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:22.049717 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c91adb0c-eeec-4813-a9ae-bb448d6e25cf tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:22.204369 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-c91adb0c-eeec-4813-a9ae-bb448d6e25cf tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:22.212477 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-c91adb0c-eeec-4813-a9ae-bb448d6e25cf tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 0.175848 Jul 19 11:05:22.213396 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 75/151] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:22 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:22.250466 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-62ee5d9f-51f1-4ef6-8806-2289268e3684 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:22.251930 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-62ee5d9f-51f1-4ef6-8806-2289268e3684 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:22.258973 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-62ee5d9f-51f1-4ef6-8806-2289268e3684 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:22.259501 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-62ee5d9f-51f1-4ef6-8806-2289268e3684 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:22.260588 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-62ee5d9f-51f1-4ef6-8806-2289268e3684 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:22.261276 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-62ee5d9f-51f1-4ef6-8806-2289268e3684 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:22.426168 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-9f752214-773c-43f7-bd2c-248474688b63 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:22.426852 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-9f752214-773c-43f7-bd2c-248474688b63 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:22.433068 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9f752214-773c-43f7-bd2c-248474688b63 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:22.433521 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9f752214-773c-43f7-bd2c-248474688b63 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:22.434275 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9f752214-773c-43f7-bd2c-248474688b63 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:22.434711 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9f752214-773c-43f7-bd2c-248474688b63 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:22.462982 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-62ee5d9f-51f1-4ef6-8806-2289268e3684 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:22.472726 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-62ee5d9f-51f1-4ef6-8806-2289268e3684 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1367 microversion: 2.1 time: 0.226314 Jul 19 11:05:22.473400 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 77/152] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:22 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1367 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:05:22.565514 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-9f752214-773c-43f7-bd2c-248474688b63 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:22.574093 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-9f752214-773c-43f7-bd2c-248474688b63 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:22.575605 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-9f752214-773c-43f7-bd2c-248474688b63 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:22.576718 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-9f752214-773c-43f7-bd2c-248474688b63 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1405 microversion: 2.20 time: 0.154449 Jul 19 11:05:22.577271 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 76/153] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:05:22 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1405 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:05:22.669771 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-f7d411db-bc83-4936-8d8c-d65e51104a55 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:22.670457 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-f7d411db-bc83-4936-8d8c-d65e51104a55 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:22.676806 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f7d411db-bc83-4936-8d8c-d65e51104a55 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:22.677279 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f7d411db-bc83-4936-8d8c-d65e51104a55 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:22.678050 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f7d411db-bc83-4936-8d8c-d65e51104a55 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:22.678484 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f7d411db-bc83-4936-8d8c-d65e51104a55 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:22.810633 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-f7d411db-bc83-4936-8d8c-d65e51104a55 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:22.818248 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-f7d411db-bc83-4936-8d8c-d65e51104a55 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1314 microversion: 2.1 time: 0.152177 Jul 19 11:05:22.818764 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 78/154] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:22 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1314 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:22.864100 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-a0f5eb77-0a19-4672-804f-85b3e211267e req-06b1d6b8-0f0c-4590-a5d2-3e1512161045 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "49e783c8-8e6d-4789-b81d-8e1ca4a550a6", "name": "network-vif-plugged", "status": "completed", "tag": "ba1dd015-6965-43ea-a7cf-059d7c358686"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:05:22.872856 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-a0f5eb77-0a19-4672-804f-85b3e211267e req-06b1d6b8-0f0c-4590-a5d2-3e1512161045 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:22.873292 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-a0f5eb77-0a19-4672-804f-85b3e211267e req-06b1d6b8-0f0c-4590-a5d2-3e1512161045 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:22.883946 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-a0f5eb77-0a19-4672-804f-85b3e211267e req-06b1d6b8-0f0c-4590-a5d2-3e1512161045 service nova] Creating event network-vif-plugged:ba1dd015-6965-43ea-a7cf-059d7c358686 for instance 49e783c8-8e6d-4789-b81d-8e1ca4a550a6 on hv-cinder-maste Jul 19 11:05:22.889685 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-a0f5eb77-0a19-4672-804f-85b3e211267e req-06b1d6b8-0f0c-4590-a5d2-3e1512161045 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029094 Jul 19 11:05:22.890190 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 77/155] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:05:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:05:22.904812 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-bd0a8aaa-c7dc-435c-af6e-fa9f7a51bb0a tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:22.905425 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-bd0a8aaa-c7dc-435c-af6e-fa9f7a51bb0a tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:22.911489 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bd0a8aaa-c7dc-435c-af6e-fa9f7a51bb0a tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:22.911917 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bd0a8aaa-c7dc-435c-af6e-fa9f7a51bb0a tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:22.912677 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bd0a8aaa-c7dc-435c-af6e-fa9f7a51bb0a tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:22.913134 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bd0a8aaa-c7dc-435c-af6e-fa9f7a51bb0a tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:22.977379 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-bd0a8aaa-c7dc-435c-af6e-fa9f7a51bb0a tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:22.985851 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-bd0a8aaa-c7dc-435c-af6e-fa9f7a51bb0a tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1265 microversion: 2.1 time: 0.084882 Jul 19 11:05:22.997690 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 79/156] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:22 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1265 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:23.229903 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-fd79475a-8591-4083-ad05-b7c4452851d0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:23.230610 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-fd79475a-8591-4083-ad05-b7c4452851d0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:23.237829 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fd79475a-8591-4083-ad05-b7c4452851d0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:23.238316 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fd79475a-8591-4083-ad05-b7c4452851d0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:23.239082 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fd79475a-8591-4083-ad05-b7c4452851d0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:23.239517 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fd79475a-8591-4083-ad05-b7c4452851d0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:23.376831 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-fd79475a-8591-4083-ad05-b7c4452851d0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:23.384813 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-fd79475a-8591-4083-ad05-b7c4452851d0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 0.159090 Jul 19 11:05:23.385393 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 78/157] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:23 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:23.489287 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-680f3a56-27a9-4707-9906-bae07aed58af tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:23.489927 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-680f3a56-27a9-4707-9906-bae07aed58af tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:23.496412 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-680f3a56-27a9-4707-9906-bae07aed58af tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:23.497160 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-680f3a56-27a9-4707-9906-bae07aed58af tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:23.497916 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-680f3a56-27a9-4707-9906-bae07aed58af tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:23.499400 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-680f3a56-27a9-4707-9906-bae07aed58af tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:23.594409 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-b8880df1-560f-48cd-b340-1f3ffe6ae2b9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:23.595024 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-b8880df1-560f-48cd-b340-1f3ffe6ae2b9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:23.601226 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b8880df1-560f-48cd-b340-1f3ffe6ae2b9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:23.602168 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b8880df1-560f-48cd-b340-1f3ffe6ae2b9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:23.602932 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b8880df1-560f-48cd-b340-1f3ffe6ae2b9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:23.603362 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b8880df1-560f-48cd-b340-1f3ffe6ae2b9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:23.632622 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-680f3a56-27a9-4707-9906-bae07aed58af tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:23.640126 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-680f3a56-27a9-4707-9906-bae07aed58af tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1367 microversion: 2.1 time: 0.154592 Jul 19 11:05:23.640922 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 80/158] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:23 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1367 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:23.735773 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-b8880df1-560f-48cd-b340-1f3ffe6ae2b9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:23.744301 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-b8880df1-560f-48cd-b340-1f3ffe6ae2b9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:23.745842 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-b8880df1-560f-48cd-b340-1f3ffe6ae2b9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:23.747182 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-b8880df1-560f-48cd-b340-1f3ffe6ae2b9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1405 microversion: 2.20 time: 0.156677 Jul 19 11:05:23.747737 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 79/159] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:05:23 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1405 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:05:23.835159 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-9d7dcabb-a254-4aea-b196-b183498bb794 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:23.835798 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-9d7dcabb-a254-4aea-b196-b183498bb794 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:23.841694 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9d7dcabb-a254-4aea-b196-b183498bb794 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:23.842136 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9d7dcabb-a254-4aea-b196-b183498bb794 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:23.842880 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9d7dcabb-a254-4aea-b196-b183498bb794 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:23.843308 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9d7dcabb-a254-4aea-b196-b183498bb794 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:23.845800 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=100 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:05:23.969115 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-9d7dcabb-a254-4aea-b196-b183498bb794 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:23.976425 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-9d7dcabb-a254-4aea-b196-b183498bb794 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1314 microversion: 2.1 time: 0.145068 Jul 19 11:05:23.977645 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 81/160] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:23 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1314 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:24.003012 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-7339f56a-4a77-41f3-9b36-e4bd520e994e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:24.003556 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-7339f56a-4a77-41f3-9b36-e4bd520e994e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:24.009555 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7339f56a-4a77-41f3-9b36-e4bd520e994e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:24.010000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7339f56a-4a77-41f3-9b36-e4bd520e994e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:24.010751 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7339f56a-4a77-41f3-9b36-e4bd520e994e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:24.011183 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7339f56a-4a77-41f3-9b36-e4bd520e994e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:24.074069 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-7339f56a-4a77-41f3-9b36-e4bd520e994e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:24.085412 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-7339f56a-4a77-41f3-9b36-e4bd520e994e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1265 microversion: 2.1 time: 0.086234 Jul 19 11:05:24.086517 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 80/161] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:23 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1265 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:24.401763 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-fdc1c526-baf0-49e0-bc24-6e45f24dde45 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:24.402366 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-fdc1c526-baf0-49e0-bc24-6e45f24dde45 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:24.408298 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fdc1c526-baf0-49e0-bc24-6e45f24dde45 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:24.408788 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fdc1c526-baf0-49e0-bc24-6e45f24dde45 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:24.409544 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fdc1c526-baf0-49e0-bc24-6e45f24dde45 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:24.409976 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fdc1c526-baf0-49e0-bc24-6e45f24dde45 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:24.538752 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-fdc1c526-baf0-49e0-bc24-6e45f24dde45 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:24.546159 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-fdc1c526-baf0-49e0-bc24-6e45f24dde45 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 0.148868 Jul 19 11:05:24.546724 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 82/162] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:24 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:24.659277 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-866aa2bc-857c-43ac-a4ba-d5782653bcc8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:24.660253 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-866aa2bc-857c-43ac-a4ba-d5782653bcc8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:24.668808 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-866aa2bc-857c-43ac-a4ba-d5782653bcc8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:24.669460 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-866aa2bc-857c-43ac-a4ba-d5782653bcc8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:24.670588 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-866aa2bc-857c-43ac-a4ba-d5782653bcc8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:24.671228 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-866aa2bc-857c-43ac-a4ba-d5782653bcc8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:24.765486 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-75a881a1-4b02-4479-85fa-aa1fb149000b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:24.766097 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-75a881a1-4b02-4479-85fa-aa1fb149000b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:24.772089 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-75a881a1-4b02-4479-85fa-aa1fb149000b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:24.772609 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-75a881a1-4b02-4479-85fa-aa1fb149000b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:24.773364 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-75a881a1-4b02-4479-85fa-aa1fb149000b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:24.773799 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-75a881a1-4b02-4479-85fa-aa1fb149000b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:24.818952 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-866aa2bc-857c-43ac-a4ba-d5782653bcc8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:24.827609 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-866aa2bc-857c-43ac-a4ba-d5782653bcc8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1367 microversion: 2.1 time: 0.174358 Jul 19 11:05:24.828731 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 81/163] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:24 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1367 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:24.912908 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-75a881a1-4b02-4479-85fa-aa1fb149000b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:24.921205 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-75a881a1-4b02-4479-85fa-aa1fb149000b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:24.923371 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-75a881a1-4b02-4479-85fa-aa1fb149000b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:24.924500 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-75a881a1-4b02-4479-85fa-aa1fb149000b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1405 microversion: 2.20 time: 0.163293 Jul 19 11:05:24.925091 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 83/164] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:05:24 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1405 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:05:24.994425 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-c227a396-a43f-47f2-bb78-8f10cc2911e0 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:24.995033 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-c227a396-a43f-47f2-bb78-8f10cc2911e0 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:24.998274 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_api:SELECT=100 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:05:25.002176 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c227a396-a43f-47f2-bb78-8f10cc2911e0 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:25.002624 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c227a396-a43f-47f2-bb78-8f10cc2911e0 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:25.003375 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c227a396-a43f-47f2-bb78-8f10cc2911e0 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:25.003810 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c227a396-a43f-47f2-bb78-8f10cc2911e0 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:25.103694 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-6923147f-8771-4803-886a-93b7db41d15c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:25.104327 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-6923147f-8771-4803-886a-93b7db41d15c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:25.110827 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6923147f-8771-4803-886a-93b7db41d15c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:25.111341 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6923147f-8771-4803-886a-93b7db41d15c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:25.112246 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6923147f-8771-4803-886a-93b7db41d15c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:25.112680 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6923147f-8771-4803-886a-93b7db41d15c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:25.137842 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-c227a396-a43f-47f2-bb78-8f10cc2911e0 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:25.145700 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-c227a396-a43f-47f2-bb78-8f10cc2911e0 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1314 microversion: 2.1 time: 0.155596 Jul 19 11:05:25.146714 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 82/165] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:24 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1314 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:25.174271 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-6923147f-8771-4803-886a-93b7db41d15c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:25.182209 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-6923147f-8771-4803-886a-93b7db41d15c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1265 microversion: 2.1 time: 0.082967 Jul 19 11:05:25.182764 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 84/166] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:25 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1265 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:25.562711 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-21d2cd59-d369-4c9f-a686-9b443ddc094a tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:25.563432 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-21d2cd59-d369-4c9f-a686-9b443ddc094a tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:25.569971 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-21d2cd59-d369-4c9f-a686-9b443ddc094a tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:25.570435 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-21d2cd59-d369-4c9f-a686-9b443ddc094a tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:25.571181 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-21d2cd59-d369-4c9f-a686-9b443ddc094a tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:25.571609 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-21d2cd59-d369-4c9f-a686-9b443ddc094a tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:25.701021 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-21d2cd59-d369-4c9f-a686-9b443ddc094a tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:25.708453 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-21d2cd59-d369-4c9f-a686-9b443ddc094a tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 0.149641 Jul 19 11:05:25.709826 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 83/167] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:25 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:25.844646 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-3c103321-00a4-4dd9-9a05-8bbd71f4f706 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:25.845271 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-3c103321-00a4-4dd9-9a05-8bbd71f4f706 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:25.851075 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3c103321-00a4-4dd9-9a05-8bbd71f4f706 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:25.851526 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3c103321-00a4-4dd9-9a05-8bbd71f4f706 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:25.852278 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3c103321-00a4-4dd9-9a05-8bbd71f4f706 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:25.852722 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3c103321-00a4-4dd9-9a05-8bbd71f4f706 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:25.944920 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-8f9abb65-a15a-4ba5-bdf5-ee5429f5bd58 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:25.946009 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-8f9abb65-a15a-4ba5-bdf5-ee5429f5bd58 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:25.957618 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8f9abb65-a15a-4ba5-bdf5-ee5429f5bd58 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:25.958301 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8f9abb65-a15a-4ba5-bdf5-ee5429f5bd58 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:25.958966 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8f9abb65-a15a-4ba5-bdf5-ee5429f5bd58 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:25.959492 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8f9abb65-a15a-4ba5-bdf5-ee5429f5bd58 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:26.006309 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-3c103321-00a4-4dd9-9a05-8bbd71f4f706 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:26.014840 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-3c103321-00a4-4dd9-9a05-8bbd71f4f706 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1367 microversion: 2.1 time: 0.174029 Jul 19 11:05:26.015643 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 85/168] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:25 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1367 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:26.093188 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-8f9abb65-a15a-4ba5-bdf5-ee5429f5bd58 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:26.101023 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-8f9abb65-a15a-4ba5-bdf5-ee5429f5bd58 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:26.102555 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-8f9abb65-a15a-4ba5-bdf5-ee5429f5bd58 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:26.103631 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-8f9abb65-a15a-4ba5-bdf5-ee5429f5bd58 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1405 microversion: 2.20 time: 0.164320 Jul 19 11:05:26.104179 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 84/169] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:05:25 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1405 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:05:26.162258 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-88c0b670-862f-4223-9db0-f7ce743cb8ed tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:26.163084 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-88c0b670-862f-4223-9db0-f7ce743cb8ed tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:26.169653 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-88c0b670-862f-4223-9db0-f7ce743cb8ed tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:26.170091 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-88c0b670-862f-4223-9db0-f7ce743cb8ed tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:26.170837 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-88c0b670-862f-4223-9db0-f7ce743cb8ed tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:26.171267 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-88c0b670-862f-4223-9db0-f7ce743cb8ed tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:26.199793 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-7d62a16e-d400-4878-906e-6204ba6e0f3c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:26.200501 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-7d62a16e-d400-4878-906e-6204ba6e0f3c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:26.207314 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7d62a16e-d400-4878-906e-6204ba6e0f3c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:26.207755 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7d62a16e-d400-4878-906e-6204ba6e0f3c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:26.208512 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7d62a16e-d400-4878-906e-6204ba6e0f3c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:26.208964 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7d62a16e-d400-4878-906e-6204ba6e0f3c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:26.278581 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-7d62a16e-d400-4878-906e-6204ba6e0f3c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:26.286387 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-7d62a16e-d400-4878-906e-6204ba6e0f3c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1273 microversion: 2.1 time: 0.091519 Jul 19 11:05:26.287148 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 85/170] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:26 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1273 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:26.311346 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-88c0b670-862f-4223-9db0-f7ce743cb8ed tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:26.319243 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-88c0b670-862f-4223-9db0-f7ce743cb8ed tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1502 microversion: 2.1 time: 0.161039 Jul 19 11:05:26.319822 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 86/171] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:26 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1502 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:26.728447 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-c73a2c09-e0cc-410b-85e9-7b41206bbeee tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:26.728998 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-c73a2c09-e0cc-410b-85e9-7b41206bbeee tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:26.735576 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c73a2c09-e0cc-410b-85e9-7b41206bbeee tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:26.736143 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c73a2c09-e0cc-410b-85e9-7b41206bbeee tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:26.737507 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c73a2c09-e0cc-410b-85e9-7b41206bbeee tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:26.737954 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c73a2c09-e0cc-410b-85e9-7b41206bbeee tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:26.882430 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-c73a2c09-e0cc-410b-85e9-7b41206bbeee tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:26.889924 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-c73a2c09-e0cc-410b-85e9-7b41206bbeee tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 0.166365 Jul 19 11:05:26.890488 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 86/172] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:26 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:27.032928 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-241cb300-e055-4746-8e94-87cfeeb5910b tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:27.033630 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-241cb300-e055-4746-8e94-87cfeeb5910b tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:27.040171 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-241cb300-e055-4746-8e94-87cfeeb5910b tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:27.040672 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-241cb300-e055-4746-8e94-87cfeeb5910b tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:27.041429 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-241cb300-e055-4746-8e94-87cfeeb5910b tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:27.041870 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-241cb300-e055-4746-8e94-87cfeeb5910b tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:27.121686 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-e359126d-310b-443a-95fc-9db66edfb4db tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:27.122568 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-e359126d-310b-443a-95fc-9db66edfb4db tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:27.128822 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e359126d-310b-443a-95fc-9db66edfb4db tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:27.129275 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e359126d-310b-443a-95fc-9db66edfb4db tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:27.130068 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e359126d-310b-443a-95fc-9db66edfb4db tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:27.130501 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e359126d-310b-443a-95fc-9db66edfb4db tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:27.199349 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-241cb300-e055-4746-8e94-87cfeeb5910b tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:27.208164 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-241cb300-e055-4746-8e94-87cfeeb5910b tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1367 microversion: 2.1 time: 0.179136 Jul 19 11:05:27.212782 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 87/173] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:27 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1367 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:27.295349 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-e359126d-310b-443a-95fc-9db66edfb4db tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:27.303661 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-e359126d-310b-443a-95fc-9db66edfb4db tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:27.304354 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-d83016d1-df8a-47dc-9090-a429a7fdaa2d tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:27.305146 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-d83016d1-df8a-47dc-9090-a429a7fdaa2d tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:27.305498 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-e359126d-310b-443a-95fc-9db66edfb4db tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:27.306796 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-e359126d-310b-443a-95fc-9db66edfb4db tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1405 microversion: 2.20 time: 0.188841 Jul 19 11:05:27.311949 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d83016d1-df8a-47dc-9090-a429a7fdaa2d tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:27.312394 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d83016d1-df8a-47dc-9090-a429a7fdaa2d tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:27.313225 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d83016d1-df8a-47dc-9090-a429a7fdaa2d tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:27.313650 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d83016d1-df8a-47dc-9090-a429a7fdaa2d tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:27.317541 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 87/174] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:05:27 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1405 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:05:27.382445 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-d83016d1-df8a-47dc-9090-a429a7fdaa2d tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:27.392061 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-d83016d1-df8a-47dc-9090-a429a7fdaa2d tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1283 microversion: 2.1 time: 0.092266 Jul 19 11:05:27.392621 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 88/175] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:27 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1283 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:27.716339 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-f3942053-d2e4-4d71-bcfc-e8ab61728e42 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "04d57cb1-d412-42a2-9982-8d19bc64b939"}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:05:27.717483 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-f3942053-d2e4-4d71-bcfc-e8ab61728e42 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:27.723535 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f3942053-d2e4-4d71-bcfc-e8ab61728e42 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:27.723978 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f3942053-d2e4-4d71-bcfc-e8ab61728e42 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:27.724755 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f3942053-d2e4-4d71-bcfc-e8ab61728e42 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:27.725190 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f3942053-d2e4-4d71-bcfc-e8ab61728e42 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:27.906595 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-97ead50a-211b-4f5d-ad43-e3a6a207a6f9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:27.907377 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-97ead50a-211b-4f5d-ad43-e3a6a207a6f9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:27.909652 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:05:27.915458 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-97ead50a-211b-4f5d-ad43-e3a6a207a6f9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:27.916414 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-97ead50a-211b-4f5d-ad43-e3a6a207a6f9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:27.917648 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-97ead50a-211b-4f5d-ad43-e3a6a207a6f9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:27.918259 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-97ead50a-211b-4f5d-ad43-e3a6a207a6f9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:28.075792 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-97ead50a-211b-4f5d-ad43-e3a6a207a6f9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:28.083810 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-97ead50a-211b-4f5d-ad43-e3a6a207a6f9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 0.181238 Jul 19 11:05:28.084540 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 89/176] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:27 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:05:28.126416 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-566db8bf-54c7-4441-a416-bba830e036d6 req-79d7d5d0-292a-489c-bb66-bbd5c5f79dec service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f52fbfae-577b-403d-a4d3-9a4b79ae2326", "name": "network-vif-plugged", "status": "completed", "tag": "748a7315-79e9-4ab8-ad6a-336c057e8b2c"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:05:28.134563 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-566db8bf-54c7-4441-a416-bba830e036d6 req-79d7d5d0-292a-489c-bb66-bbd5c5f79dec service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:28.135097 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-566db8bf-54c7-4441-a416-bba830e036d6 req-79d7d5d0-292a-489c-bb66-bbd5c5f79dec service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:28.152087 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-566db8bf-54c7-4441-a416-bba830e036d6 req-79d7d5d0-292a-489c-bb66-bbd5c5f79dec service nova] Creating event network-vif-plugged:748a7315-79e9-4ab8-ad6a-336c057e8b2c for instance f52fbfae-577b-403d-a4d3-9a4b79ae2326 on hv-cinder-maste Jul 19 11:05:28.158480 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-566db8bf-54c7-4441-a416-bba830e036d6 req-79d7d5d0-292a-489c-bb66-bbd5c5f79dec service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036170 Jul 19 11:05:28.159098 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 90/177] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:05:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:05:28.226207 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-c062724e-9544-4c0b-891c-cbd881eb2e6a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:28.227460 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-c062724e-9544-4c0b-891c-cbd881eb2e6a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:28.233980 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c062724e-9544-4c0b-891c-cbd881eb2e6a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:28.234433 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c062724e-9544-4c0b-891c-cbd881eb2e6a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:28.235189 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c062724e-9544-4c0b-891c-cbd881eb2e6a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:28.235620 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c062724e-9544-4c0b-891c-cbd881eb2e6a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:28.361520 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-c062724e-9544-4c0b-891c-cbd881eb2e6a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:28.369561 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-c062724e-9544-4c0b-891c-cbd881eb2e6a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1367 microversion: 2.1 time: 0.147609 Jul 19 11:05:28.370059 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 91/178] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:28 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1367 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:05:28.374721 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-f5955dc5-7417-4da6-a88f-d9ea9f446dc5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:28.375319 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-f5955dc5-7417-4da6-a88f-d9ea9f446dc5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:28.381544 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f5955dc5-7417-4da6-a88f-d9ea9f446dc5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:28.381991 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f5955dc5-7417-4da6-a88f-d9ea9f446dc5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:28.382750 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f5955dc5-7417-4da6-a88f-d9ea9f446dc5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:28.383217 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f5955dc5-7417-4da6-a88f-d9ea9f446dc5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:28.508128 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-f5955dc5-7417-4da6-a88f-d9ea9f446dc5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:28.515962 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-f5955dc5-7417-4da6-a88f-d9ea9f446dc5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:28.517554 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-f5955dc5-7417-4da6-a88f-d9ea9f446dc5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:28.518654 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-f5955dc5-7417-4da6-a88f-d9ea9f446dc5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1405 microversion: 2.20 time: 0.147669 Jul 19 11:05:28.519202 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 92/179] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:05:28 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1405 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:05:28.523930 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-698a4316-635b-4afb-9733-37ef1a12fcb7 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:28.524522 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-698a4316-635b-4afb-9733-37ef1a12fcb7 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:28.530050 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-698a4316-635b-4afb-9733-37ef1a12fcb7 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:28.530492 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-698a4316-635b-4afb-9733-37ef1a12fcb7 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:28.531238 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-698a4316-635b-4afb-9733-37ef1a12fcb7 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:28.531663 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-698a4316-635b-4afb-9733-37ef1a12fcb7 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:28.593690 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-698a4316-635b-4afb-9733-37ef1a12fcb7 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:28.600896 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-698a4316-635b-4afb-9733-37ef1a12fcb7 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1271 microversion: 2.1 time: 0.080727 Jul 19 11:05:28.602088 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 93/180] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:28 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1271 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:29.100991 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-002311dd-3820-462f-ac45-f1365f89bd92 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:29.101619 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-002311dd-3820-462f-ac45-f1365f89bd92 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:29.107625 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-002311dd-3820-462f-ac45-f1365f89bd92 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:29.108069 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-002311dd-3820-462f-ac45-f1365f89bd92 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:29.108852 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-002311dd-3820-462f-ac45-f1365f89bd92 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:29.109282 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-002311dd-3820-462f-ac45-f1365f89bd92 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:29.235980 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-002311dd-3820-462f-ac45-f1365f89bd92 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:29.244076 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-002311dd-3820-462f-ac45-f1365f89bd92 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 0.147164 Jul 19 11:05:29.244655 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 94/181] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:29 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:05:29.386817 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-cc1964d4-77d7-45c7-b83f-160fd1a7ca39 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:29.387397 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-cc1964d4-77d7-45c7-b83f-160fd1a7ca39 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:29.393364 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cc1964d4-77d7-45c7-b83f-160fd1a7ca39 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:29.393801 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cc1964d4-77d7-45c7-b83f-160fd1a7ca39 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:29.394547 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cc1964d4-77d7-45c7-b83f-160fd1a7ca39 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:29.394974 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cc1964d4-77d7-45c7-b83f-160fd1a7ca39 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:29.519872 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-cc1964d4-77d7-45c7-b83f-160fd1a7ca39 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:29.527158 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-cc1964d4-77d7-45c7-b83f-160fd1a7ca39 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1367 microversion: 2.1 time: 0.144723 Jul 19 11:05:29.527710 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 95/182] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:29 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1367 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:05:29.536326 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-21fbc1ba-ccd2-443d-815b-6608399c1f8e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:29.536943 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-21fbc1ba-ccd2-443d-815b-6608399c1f8e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:29.538670 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell0:SELECT=2 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:05:29.544338 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-21fbc1ba-ccd2-443d-815b-6608399c1f8e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:29.544837 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-21fbc1ba-ccd2-443d-815b-6608399c1f8e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:29.545596 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-21fbc1ba-ccd2-443d-815b-6608399c1f8e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:29.546143 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-21fbc1ba-ccd2-443d-815b-6608399c1f8e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:29.561903 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell0:SELECT=6 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:05:29.676041 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-21fbc1ba-ccd2-443d-815b-6608399c1f8e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:29.684614 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-21fbc1ba-ccd2-443d-815b-6608399c1f8e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:29.686145 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-21fbc1ba-ccd2-443d-815b-6608399c1f8e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:29.687224 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-21fbc1ba-ccd2-443d-815b-6608399c1f8e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1405 microversion: 2.20 time: 0.154477 Jul 19 11:05:29.687365 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell0:SELECT=2 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:05:29.687785 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 96/183] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:05:29 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1405 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:05:29.693883 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-ed7ee9b1-030a-48b3-8737-b109fe4f6ddb tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:29.694493 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-ed7ee9b1-030a-48b3-8737-b109fe4f6ddb tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:29.700337 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ed7ee9b1-030a-48b3-8737-b109fe4f6ddb tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:29.700822 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ed7ee9b1-030a-48b3-8737-b109fe4f6ddb tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:29.701586 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ed7ee9b1-030a-48b3-8737-b109fe4f6ddb tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:29.702020 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ed7ee9b1-030a-48b3-8737-b109fe4f6ddb tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:29.762335 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-ed7ee9b1-030a-48b3-8737-b109fe4f6ddb tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:29.769745 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-ed7ee9b1-030a-48b3-8737-b109fe4f6ddb tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1271 microversion: 2.1 time: 0.080235 Jul 19 11:05:29.770252 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 97/184] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:29 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1271 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:30.261038 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-02ca7ad1-3ccd-473a-9d6c-89d4b5b5d937 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:30.261638 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-02ca7ad1-3ccd-473a-9d6c-89d4b5b5d937 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:30.267737 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-02ca7ad1-3ccd-473a-9d6c-89d4b5b5d937 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:30.268184 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-02ca7ad1-3ccd-473a-9d6c-89d4b5b5d937 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:30.269008 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-02ca7ad1-3ccd-473a-9d6c-89d4b5b5d937 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:30.269443 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-02ca7ad1-3ccd-473a-9d6c-89d4b5b5d937 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:30.284742 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=100 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:05:30.397710 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-02ca7ad1-3ccd-473a-9d6c-89d4b5b5d937 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:30.405722 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-02ca7ad1-3ccd-473a-9d6c-89d4b5b5d937 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 0.148513 Jul 19 11:05:30.406283 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 98/185] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:30 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:05:30.543227 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-f05290ad-9223-491f-b93e-5ea40f63d9c4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:30.543764 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-f05290ad-9223-491f-b93e-5ea40f63d9c4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:30.550109 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f05290ad-9223-491f-b93e-5ea40f63d9c4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:30.550578 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f05290ad-9223-491f-b93e-5ea40f63d9c4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:30.551335 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f05290ad-9223-491f-b93e-5ea40f63d9c4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:30.551762 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f05290ad-9223-491f-b93e-5ea40f63d9c4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:30.701863 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-f05290ad-9223-491f-b93e-5ea40f63d9c4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:30.709269 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-f05290ad-9223-491f-b93e-5ea40f63d9c4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1367 microversion: 2.1 time: 0.169837 Jul 19 11:05:30.710060 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 99/186] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:30 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1367 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:05:30.715955 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-01490196-6a24-4a48-9dbc-2f72665a92e9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:30.716556 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-01490196-6a24-4a48-9dbc-2f72665a92e9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:30.724554 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-01490196-6a24-4a48-9dbc-2f72665a92e9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:30.725051 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-01490196-6a24-4a48-9dbc-2f72665a92e9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:30.725820 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-01490196-6a24-4a48-9dbc-2f72665a92e9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:30.726258 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-01490196-6a24-4a48-9dbc-2f72665a92e9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:30.856084 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-01490196-6a24-4a48-9dbc-2f72665a92e9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:30.863925 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-01490196-6a24-4a48-9dbc-2f72665a92e9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:30.865465 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-01490196-6a24-4a48-9dbc-2f72665a92e9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:30.866539 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-01490196-6a24-4a48-9dbc-2f72665a92e9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1405 microversion: 2.20 time: 0.155449 Jul 19 11:05:30.867587 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 100/187] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:05:30 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1405 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:05:30.872218 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-474ce5d8-c58e-4006-902e-53fd9ce6da1c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:30.872842 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-474ce5d8-c58e-4006-902e-53fd9ce6da1c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:30.878357 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-474ce5d8-c58e-4006-902e-53fd9ce6da1c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:30.878798 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-474ce5d8-c58e-4006-902e-53fd9ce6da1c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:30.879543 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-474ce5d8-c58e-4006-902e-53fd9ce6da1c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:30.879969 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-474ce5d8-c58e-4006-902e-53fd9ce6da1c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:30.946539 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-474ce5d8-c58e-4006-902e-53fd9ce6da1c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:30.951805 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-474ce5d8-c58e-4006-902e-53fd9ce6da1c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1271 microversion: 2.1 time: 0.083213 Jul 19 11:05:30.952391 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 101/188] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:30 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1271 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:31.423385 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-67800dfd-4395-48de-a4ac-aa1320ffe2cf tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:31.424965 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-67800dfd-4395-48de-a4ac-aa1320ffe2cf tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:31.433313 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-67800dfd-4395-48de-a4ac-aa1320ffe2cf tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:31.434092 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-67800dfd-4395-48de-a4ac-aa1320ffe2cf tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:31.435132 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-67800dfd-4395-48de-a4ac-aa1320ffe2cf tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:31.435753 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-67800dfd-4395-48de-a4ac-aa1320ffe2cf tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:31.644726 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-67800dfd-4395-48de-a4ac-aa1320ffe2cf tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:31.653199 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-67800dfd-4395-48de-a4ac-aa1320ffe2cf tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 0.234622 Jul 19 11:05:31.653990 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 102/189] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:31 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:05:31.727953 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-ac6cbe7b-e4c1-4f3b-86f0-43896d4fe009 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:31.728560 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-ac6cbe7b-e4c1-4f3b-86f0-43896d4fe009 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:31.734555 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ac6cbe7b-e4c1-4f3b-86f0-43896d4fe009 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:31.735175 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ac6cbe7b-e4c1-4f3b-86f0-43896d4fe009 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:31.735933 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ac6cbe7b-e4c1-4f3b-86f0-43896d4fe009 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:31.736380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ac6cbe7b-e4c1-4f3b-86f0-43896d4fe009 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:31.944262 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-ac6cbe7b-e4c1-4f3b-86f0-43896d4fe009 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:31.953256 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-ac6cbe7b-e4c1-4f3b-86f0-43896d4fe009 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1367 microversion: 2.1 time: 0.229555 Jul 19 11:05:31.954225 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 103/190] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:31 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1367 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:05:31.960457 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-4abc9abd-7271-4c00-afa4-e7ef10e26f4f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:31.961861 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-4abc9abd-7271-4c00-afa4-e7ef10e26f4f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:31.971041 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4abc9abd-7271-4c00-afa4-e7ef10e26f4f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:31.971836 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4abc9abd-7271-4c00-afa4-e7ef10e26f4f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:31.973717 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4abc9abd-7271-4c00-afa4-e7ef10e26f4f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:31.974256 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4abc9abd-7271-4c00-afa4-e7ef10e26f4f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:32.109116 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-4abc9abd-7271-4c00-afa4-e7ef10e26f4f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:32.117110 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-4abc9abd-7271-4c00-afa4-e7ef10e26f4f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:32.118631 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-4abc9abd-7271-4c00-afa4-e7ef10e26f4f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:32.119737 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-4abc9abd-7271-4c00-afa4-e7ef10e26f4f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1405 microversion: 2.20 time: 0.163556 Jul 19 11:05:32.120297 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 104/191] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:05:31 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1405 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:05:32.125050 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-826262cd-9113-4946-a1f3-d257b092da24 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:32.125584 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-826262cd-9113-4946-a1f3-d257b092da24 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:32.131194 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-826262cd-9113-4946-a1f3-d257b092da24 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:32.131648 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-826262cd-9113-4946-a1f3-d257b092da24 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:32.132415 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-826262cd-9113-4946-a1f3-d257b092da24 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:32.133369 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-826262cd-9113-4946-a1f3-d257b092da24 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:32.270109 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-826262cd-9113-4946-a1f3-d257b092da24 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:32.277684 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-826262cd-9113-4946-a1f3-d257b092da24 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1313 microversion: 2.1 time: 0.156402 Jul 19 11:05:32.278256 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 105/192] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:32 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1313 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:32.669463 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-0c3ead71-9cb8-4b5d-883a-7a4c873211a2 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:32.670053 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-0c3ead71-9cb8-4b5d-883a-7a4c873211a2 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:32.678358 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0c3ead71-9cb8-4b5d-883a-7a4c873211a2 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:32.678851 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0c3ead71-9cb8-4b5d-883a-7a4c873211a2 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:32.680384 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0c3ead71-9cb8-4b5d-883a-7a4c873211a2 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:32.680791 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0c3ead71-9cb8-4b5d-883a-7a4c873211a2 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:32.758626 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-f3942053-d2e4-4d71-bcfc-e8ab61728e42 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "POST /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 5.042559 Jul 19 11:05:32.761004 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 88/193] 10.222.0.33 () {60 vars in 1322 bytes} [Tue Jul 19 11:05:27 2022] POST /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2/os-volume_attachments => generated 194 bytes in 5050 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:05:32.945452 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-0680a8f5-2471-4c50-b978-d259fbe87283 req-cb0bbdf9-64fd-46c8-93c3-9ea61c92e167 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4c71f564-de63-4fcd-88ce-01ecba3d8372", "name": "network-vif-plugged", "status": "completed", "tag": "526a30d7-5fcf-4c45-86bf-6ddbc902d48c"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:05:32.963649 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-0680a8f5-2471-4c50-b978-d259fbe87283 req-cb0bbdf9-64fd-46c8-93c3-9ea61c92e167 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:32.964358 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-0680a8f5-2471-4c50-b978-d259fbe87283 req-cb0bbdf9-64fd-46c8-93c3-9ea61c92e167 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:32.973331 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:05:32.986288 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-0680a8f5-2471-4c50-b978-d259fbe87283 req-cb0bbdf9-64fd-46c8-93c3-9ea61c92e167 service nova] Creating event network-vif-plugged:526a30d7-5fcf-4c45-86bf-6ddbc902d48c for instance 4c71f564-de63-4fcd-88ce-01ecba3d8372 on hv-cinder-maste Jul 19 11:05:32.993476 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-0680a8f5-2471-4c50-b978-d259fbe87283 req-cb0bbdf9-64fd-46c8-93c3-9ea61c92e167 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052668 Jul 19 11:05:32.994775 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 89/194] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:05:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:05:33.006980 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-d9bc38cc-1e83-4e5b-93ec-07315658337a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:33.008077 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-d9bc38cc-1e83-4e5b-93ec-07315658337a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:33.013185 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-0c3ead71-9cb8-4b5d-883a-7a4c873211a2 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:33.022012 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d9bc38cc-1e83-4e5b-93ec-07315658337a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:33.022936 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d9bc38cc-1e83-4e5b-93ec-07315658337a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:33.024569 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d9bc38cc-1e83-4e5b-93ec-07315658337a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:33.025037 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d9bc38cc-1e83-4e5b-93ec-07315658337a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:33.025350 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-0c3ead71-9cb8-4b5d-883a-7a4c873211a2 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 0.359717 Jul 19 11:05:33.026909 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 106/195] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:32 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:05:33.139586 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-708481f0-70c4-4773-880c-39604e0313e8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:33.140713 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-708481f0-70c4-4773-880c-39604e0313e8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:33.149105 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-708481f0-70c4-4773-880c-39604e0313e8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:33.149909 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-708481f0-70c4-4773-880c-39604e0313e8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:33.150702 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-708481f0-70c4-4773-880c-39604e0313e8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:33.151199 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-708481f0-70c4-4773-880c-39604e0313e8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:33.274204 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-d9bc38cc-1e83-4e5b-93ec-07315658337a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:33.281839 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-d9bc38cc-1e83-4e5b-93ec-07315658337a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1367 microversion: 2.1 time: 0.285851 Jul 19 11:05:33.282692 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 90/196] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:32 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1367 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:33.295481 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-889394bc-4969-4d5a-be8f-8a9d6950b0f8 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:33.296391 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-889394bc-4969-4d5a-be8f-8a9d6950b0f8 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:33.302695 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-889394bc-4969-4d5a-be8f-8a9d6950b0f8 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:33.303330 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-889394bc-4969-4d5a-be8f-8a9d6950b0f8 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:33.304273 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-889394bc-4969-4d5a-be8f-8a9d6950b0f8 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:33.304762 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-889394bc-4969-4d5a-be8f-8a9d6950b0f8 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:33.359848 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-708481f0-70c4-4773-880c-39604e0313e8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:33.368826 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-708481f0-70c4-4773-880c-39604e0313e8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:33.370619 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-708481f0-70c4-4773-880c-39604e0313e8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:33.372198 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-708481f0-70c4-4773-880c-39604e0313e8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1405 microversion: 2.20 time: 0.236963 Jul 19 11:05:33.372843 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 107/197] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:05:33 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1405 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:05:33.413776 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-83d316bf-dc7f-4d17-82b1-245b90b7254d req-5094f261-0843-406c-9787-40fa8f7cf6c4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b4e0099f-88fa-4929-94a2-ef54325bace7", "tag": "76b1d057-9d32-40d9-8cd9-1ba190a6841d"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:05:33.422152 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-83d316bf-dc7f-4d17-82b1-245b90b7254d req-5094f261-0843-406c-9787-40fa8f7cf6c4 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:33.422728 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-83d316bf-dc7f-4d17-82b1-245b90b7254d req-5094f261-0843-406c-9787-40fa8f7cf6c4 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:33.434521 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-83d316bf-dc7f-4d17-82b1-245b90b7254d req-5094f261-0843-406c-9787-40fa8f7cf6c4 service nova] Creating event network-changed:76b1d057-9d32-40d9-8cd9-1ba190a6841d for instance b4e0099f-88fa-4929-94a2-ef54325bace7 on hv-cinder-maste Jul 19 11:05:33.439598 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-83d316bf-dc7f-4d17-82b1-245b90b7254d req-5094f261-0843-406c-9787-40fa8f7cf6c4 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030209 Jul 19 11:05:33.440391 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 108/198] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:05:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:05:33.445906 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-889394bc-4969-4d5a-be8f-8a9d6950b0f8 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:33.453366 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-889394bc-4969-4d5a-be8f-8a9d6950b0f8 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1313 microversion: 2.1 time: 0.161863 Jul 19 11:05:33.453952 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 91/199] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:33 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1313 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:34.042662 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-76f640a7-0f46-4dff-9361-7090e88528f0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:34.043479 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-76f640a7-0f46-4dff-9361-7090e88528f0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:34.050119 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-76f640a7-0f46-4dff-9361-7090e88528f0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:34.050566 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-76f640a7-0f46-4dff-9361-7090e88528f0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:34.051339 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-76f640a7-0f46-4dff-9361-7090e88528f0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:34.051772 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-76f640a7-0f46-4dff-9361-7090e88528f0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:34.186950 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-76f640a7-0f46-4dff-9361-7090e88528f0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:34.194331 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-76f640a7-0f46-4dff-9361-7090e88528f0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 0.156019 Jul 19 11:05:34.194885 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 109/200] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:34 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:05:34.299221 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-fab13e32-54c9-4e06-84c4-b8b4f212c7f9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:34.299825 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-fab13e32-54c9-4e06-84c4-b8b4f212c7f9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:34.306794 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fab13e32-54c9-4e06-84c4-b8b4f212c7f9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:34.307241 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fab13e32-54c9-4e06-84c4-b8b4f212c7f9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:34.308000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fab13e32-54c9-4e06-84c4-b8b4f212c7f9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:34.308435 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fab13e32-54c9-4e06-84c4-b8b4f212c7f9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:34.390658 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-6250172f-b79a-482e-9c8c-c82200de7d9b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:34.391473 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-6250172f-b79a-482e-9c8c-c82200de7d9b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:34.397546 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6250172f-b79a-482e-9c8c-c82200de7d9b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:34.398427 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6250172f-b79a-482e-9c8c-c82200de7d9b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:34.399340 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6250172f-b79a-482e-9c8c-c82200de7d9b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:34.399853 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6250172f-b79a-482e-9c8c-c82200de7d9b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:34.438633 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-fab13e32-54c9-4e06-84c4-b8b4f212c7f9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:34.448254 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-fab13e32-54c9-4e06-84c4-b8b4f212c7f9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1367 microversion: 2.1 time: 0.152762 Jul 19 11:05:34.448954 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 92/201] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:34 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1367 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:34.471147 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-0f32b057-1904-4145-9b83-ebc1510aeec4 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:34.471748 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-0f32b057-1904-4145-9b83-ebc1510aeec4 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:34.478361 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0f32b057-1904-4145-9b83-ebc1510aeec4 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:34.478805 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0f32b057-1904-4145-9b83-ebc1510aeec4 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:34.479562 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0f32b057-1904-4145-9b83-ebc1510aeec4 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:34.479994 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0f32b057-1904-4145-9b83-ebc1510aeec4 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:34.596601 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-6250172f-b79a-482e-9c8c-c82200de7d9b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:34.605607 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-6250172f-b79a-482e-9c8c-c82200de7d9b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:34.607120 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-6250172f-b79a-482e-9c8c-c82200de7d9b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:34.608409 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-6250172f-b79a-482e-9c8c-c82200de7d9b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1405 microversion: 2.20 time: 0.221511 Jul 19 11:05:34.608995 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 110/202] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:05:34 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1405 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:05:34.614816 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-0f32b057-1904-4145-9b83-ebc1510aeec4 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:34.622603 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-0f32b057-1904-4145-9b83-ebc1510aeec4 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1313 microversion: 2.1 time: 0.155053 Jul 19 11:05:34.623154 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 93/203] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:34 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1313 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:05:35.211781 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-974181b1-8a0e-4994-a5bc-27622038b8a2 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:35.212415 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-974181b1-8a0e-4994-a5bc-27622038b8a2 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:35.218418 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-974181b1-8a0e-4994-a5bc-27622038b8a2 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:35.218766 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-974181b1-8a0e-4994-a5bc-27622038b8a2 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:35.219515 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-974181b1-8a0e-4994-a5bc-27622038b8a2 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:35.219945 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-974181b1-8a0e-4994-a5bc-27622038b8a2 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:35.356257 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-974181b1-8a0e-4994-a5bc-27622038b8a2 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:35.363573 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-974181b1-8a0e-4994-a5bc-27622038b8a2 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 0.156196 Jul 19 11:05:35.364131 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 111/204] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:35 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:35.465260 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-97552bf5-b2a7-4c64-9e0b-9a7262e5dda9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:35.465866 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-97552bf5-b2a7-4c64-9e0b-9a7262e5dda9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:35.472658 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-97552bf5-b2a7-4c64-9e0b-9a7262e5dda9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:35.473174 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-97552bf5-b2a7-4c64-9e0b-9a7262e5dda9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:35.473933 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-97552bf5-b2a7-4c64-9e0b-9a7262e5dda9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:35.474361 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-97552bf5-b2a7-4c64-9e0b-9a7262e5dda9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:35.612715 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-97552bf5-b2a7-4c64-9e0b-9a7262e5dda9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:35.621808 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-97552bf5-b2a7-4c64-9e0b-9a7262e5dda9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1367 microversion: 2.1 time: 0.160870 Jul 19 11:05:35.622649 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 94/205] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:35 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1367 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:35.625594 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-5adf21d0-5958-4d95-9f0f-2b20db4abb25 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:35.626579 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-5adf21d0-5958-4d95-9f0f-2b20db4abb25 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:35.633281 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5adf21d0-5958-4d95-9f0f-2b20db4abb25 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:35.633899 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5adf21d0-5958-4d95-9f0f-2b20db4abb25 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:35.634910 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5adf21d0-5958-4d95-9f0f-2b20db4abb25 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:35.635507 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5adf21d0-5958-4d95-9f0f-2b20db4abb25 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:35.639279 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-be98d571-7c95-46b2-917d-5262c8260c91 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:35.639886 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-be98d571-7c95-46b2-917d-5262c8260c91 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:35.647071 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-be98d571-7c95-46b2-917d-5262c8260c91 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:35.647726 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-be98d571-7c95-46b2-917d-5262c8260c91 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:35.649543 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-be98d571-7c95-46b2-917d-5262c8260c91 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:35.650191 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-be98d571-7c95-46b2-917d-5262c8260c91 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:35.780412 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-5adf21d0-5958-4d95-9f0f-2b20db4abb25 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:35.791944 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-5adf21d0-5958-4d95-9f0f-2b20db4abb25 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:35.793784 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-5adf21d0-5958-4d95-9f0f-2b20db4abb25 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:35.794963 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-5adf21d0-5958-4d95-9f0f-2b20db4abb25 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1405 microversion: 2.20 time: 0.173586 Jul 19 11:05:35.795595 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 112/206] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:05:35 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1405 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:05:35.803466 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-be98d571-7c95-46b2-917d-5262c8260c91 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:35.814589 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-be98d571-7c95-46b2-917d-5262c8260c91 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1313 microversion: 2.1 time: 0.178801 Jul 19 11:05:35.815172 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 95/207] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:35 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1313 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:05:36.380076 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-efabb537-9c77-4b7f-bfce-0246f64ea552 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:36.380827 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-efabb537-9c77-4b7f-bfce-0246f64ea552 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:36.387524 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-efabb537-9c77-4b7f-bfce-0246f64ea552 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:36.387971 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-efabb537-9c77-4b7f-bfce-0246f64ea552 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:36.388821 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-efabb537-9c77-4b7f-bfce-0246f64ea552 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:36.389249 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-efabb537-9c77-4b7f-bfce-0246f64ea552 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:36.639172 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-874fd859-3e62-4bc7-ad35-4e54af5c8628 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:36.640399 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-874fd859-3e62-4bc7-ad35-4e54af5c8628 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:36.647482 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-874fd859-3e62-4bc7-ad35-4e54af5c8628 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:36.648784 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-874fd859-3e62-4bc7-ad35-4e54af5c8628 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:36.649741 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-874fd859-3e62-4bc7-ad35-4e54af5c8628 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:36.650333 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-874fd859-3e62-4bc7-ad35-4e54af5c8628 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:36.733622 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-efabb537-9c77-4b7f-bfce-0246f64ea552 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:36.740864 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-efabb537-9c77-4b7f-bfce-0246f64ea552 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1321 microversion: 2.1 time: 0.364463 Jul 19 11:05:36.741559 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 113/208] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:36 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1321 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:36.798597 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-874fd859-3e62-4bc7-ad35-4e54af5c8628 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:36.806517 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-874fd859-3e62-4bc7-ad35-4e54af5c8628 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1367 microversion: 2.1 time: 0.171132 Jul 19 11:05:36.807341 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 96/209] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:36 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1367 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:05:36.812026 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-03233cdb-2c16-4aaa-be21-69e364c95085 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:36.812623 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-03233cdb-2c16-4aaa-be21-69e364c95085 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:36.819011 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-03233cdb-2c16-4aaa-be21-69e364c95085 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:36.819655 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-03233cdb-2c16-4aaa-be21-69e364c95085 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:36.820664 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-03233cdb-2c16-4aaa-be21-69e364c95085 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:36.821263 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-03233cdb-2c16-4aaa-be21-69e364c95085 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:36.831268 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-fd01069e-c778-45d5-b3cc-bb8b5ad4b4ae tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:36.831866 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-fd01069e-c778-45d5-b3cc-bb8b5ad4b4ae tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:36.839192 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fd01069e-c778-45d5-b3cc-bb8b5ad4b4ae tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:36.839948 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fd01069e-c778-45d5-b3cc-bb8b5ad4b4ae tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:36.841088 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fd01069e-c778-45d5-b3cc-bb8b5ad4b4ae tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:36.842833 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fd01069e-c778-45d5-b3cc-bb8b5ad4b4ae tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:37.035336 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-03233cdb-2c16-4aaa-be21-69e364c95085 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:37.045598 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-03233cdb-2c16-4aaa-be21-69e364c95085 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:37.047289 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-03233cdb-2c16-4aaa-be21-69e364c95085 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:37.048538 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-03233cdb-2c16-4aaa-be21-69e364c95085 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1405 microversion: 2.20 time: 0.240432 Jul 19 11:05:37.049145 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 114/210] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:05:36 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1405 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:05:37.049286 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-fd01069e-c778-45d5-b3cc-bb8b5ad4b4ae tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:37.058542 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-fd01069e-c778-45d5-b3cc-bb8b5ad4b4ae tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1313 microversion: 2.1 time: 0.231119 Jul 19 11:05:37.061134 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 97/211] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:36 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1313 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:05:37.759345 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-cb498d26-0a67-415d-a9ab-496e103773ab tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:37.759982 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-cb498d26-0a67-415d-a9ab-496e103773ab tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:37.766051 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cb498d26-0a67-415d-a9ab-496e103773ab tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:37.766497 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cb498d26-0a67-415d-a9ab-496e103773ab tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:37.767284 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cb498d26-0a67-415d-a9ab-496e103773ab tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:37.767722 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cb498d26-0a67-415d-a9ab-496e103773ab tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:37.825669 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-d73e998c-4f88-4954-818e-c2dfc5e2fda8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:37.826373 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-d73e998c-4f88-4954-818e-c2dfc5e2fda8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:37.833170 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d73e998c-4f88-4954-818e-c2dfc5e2fda8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:37.833616 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d73e998c-4f88-4954-818e-c2dfc5e2fda8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:37.834356 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d73e998c-4f88-4954-818e-c2dfc5e2fda8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:37.834808 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d73e998c-4f88-4954-818e-c2dfc5e2fda8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:37.904655 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-cb498d26-0a67-415d-a9ab-496e103773ab tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:37.907903 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=100 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:05:37.926140 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-cb498d26-0a67-415d-a9ab-496e103773ab tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1515 microversion: 2.1 time: 0.170996 Jul 19 11:05:37.926735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 115/212] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:37 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1515 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:37.946091 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-346e94e8-c4bd-4f89-902d-b5b1c589e702 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:05:37.947251 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-346e94e8-c4bd-4f89-902d-b5b1c589e702 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:37.953622 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-346e94e8-c4bd-4f89-902d-b5b1c589e702 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:37.954073 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-346e94e8-c4bd-4f89-902d-b5b1c589e702 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:37.954832 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-346e94e8-c4bd-4f89-902d-b5b1c589e702 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:37.955426 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-346e94e8-c4bd-4f89-902d-b5b1c589e702 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:37.993893 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-d73e998c-4f88-4954-818e-c2dfc5e2fda8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:38.002421 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-d73e998c-4f88-4954-818e-c2dfc5e2fda8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1367 microversion: 2.1 time: 0.180488 Jul 19 11:05:38.002986 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 98/213] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:37 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1367 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:05:38.019213 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.objects.instance [None req-346e94e8-c4bd-4f89-902d-b5b1c589e702 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lazy-loading 'flavor' on Instance uuid 49e783c8-8e6d-4789-b81d-8e1ca4a550a6 {{(pid=117849) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 19 11:05:38.066634 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-28744ea1-77f9-4a3c-91f4-555dd968db9c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:38.067381 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-28744ea1-77f9-4a3c-91f4-555dd968db9c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:38.074885 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-28744ea1-77f9-4a3c-91f4-555dd968db9c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:38.075593 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-28744ea1-77f9-4a3c-91f4-555dd968db9c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:38.076793 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-28744ea1-77f9-4a3c-91f4-555dd968db9c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:38.077451 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-28744ea1-77f9-4a3c-91f4-555dd968db9c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:38.195976 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-346e94e8-c4bd-4f89-902d-b5b1c589e702 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "POST /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6/action" status: 200 len: 32 microversion: 2.1 time: 0.252708 Jul 19 11:05:38.197483 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 116/214] 10.222.0.33 () {60 vars in 1277 bytes} [Tue Jul 19 11:05:37 2022] POST /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6/action => generated 32 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 19 11:05:38.203150 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-7b7b7065-1c86-48bb-b29e-ae2652fbbee1 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:38.203932 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-7b7b7065-1c86-48bb-b29e-ae2652fbbee1 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:38.211574 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7b7b7065-1c86-48bb-b29e-ae2652fbbee1 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:38.212110 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7b7b7065-1c86-48bb-b29e-ae2652fbbee1 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:38.213001 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7b7b7065-1c86-48bb-b29e-ae2652fbbee1 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:38.213622 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7b7b7065-1c86-48bb-b29e-ae2652fbbee1 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:38.249008 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-28744ea1-77f9-4a3c-91f4-555dd968db9c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:38.259919 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-28744ea1-77f9-4a3c-91f4-555dd968db9c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:38.261702 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-28744ea1-77f9-4a3c-91f4-555dd968db9c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:38.263321 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-28744ea1-77f9-4a3c-91f4-555dd968db9c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1405 microversion: 2.20 time: 0.200771 Jul 19 11:05:38.264039 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 99/215] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:05:38 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1405 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:05:38.268779 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-65f2b069-65c3-4fcf-9d60-7c3033f863c6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:38.269544 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-65f2b069-65c3-4fcf-9d60-7c3033f863c6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:38.276920 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-65f2b069-65c3-4fcf-9d60-7c3033f863c6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:38.277534 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-65f2b069-65c3-4fcf-9d60-7c3033f863c6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:38.278472 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-65f2b069-65c3-4fcf-9d60-7c3033f863c6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:38.279049 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-65f2b069-65c3-4fcf-9d60-7c3033f863c6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:38.402866 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-7b7b7065-1c86-48bb-b29e-ae2652fbbee1 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:38.415416 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-7b7b7065-1c86-48bb-b29e-ae2652fbbee1 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1313 microversion: 2.1 time: 0.213495 Jul 19 11:05:38.415416 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 117/216] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:38 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1313 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:38.470773 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-65f2b069-65c3-4fcf-9d60-7c3033f863c6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:38.478354 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-65f2b069-65c3-4fcf-9d60-7c3033f863c6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.213054 Jul 19 11:05:38.479108 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 100/217] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:38 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:39.021808 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-4d02de91-8cd5-4f3a-b919-7d6f1182d0eb tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:39.022494 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-4d02de91-8cd5-4f3a-b919-7d6f1182d0eb tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:39.029204 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4d02de91-8cd5-4f3a-b919-7d6f1182d0eb tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:39.029655 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4d02de91-8cd5-4f3a-b919-7d6f1182d0eb tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:39.030417 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4d02de91-8cd5-4f3a-b919-7d6f1182d0eb tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:39.030849 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4d02de91-8cd5-4f3a-b919-7d6f1182d0eb tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:39.163234 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-4d02de91-8cd5-4f3a-b919-7d6f1182d0eb tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:39.171616 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-4d02de91-8cd5-4f3a-b919-7d6f1182d0eb tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1367 microversion: 2.1 time: 0.153599 Jul 19 11:05:39.172262 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 118/218] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:39 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1367 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:39.283296 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-40d70c6b-4902-4aa9-96dc-a06571bae0d1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:39.283905 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-40d70c6b-4902-4aa9-96dc-a06571bae0d1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:39.291038 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-40d70c6b-4902-4aa9-96dc-a06571bae0d1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:39.291487 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-40d70c6b-4902-4aa9-96dc-a06571bae0d1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:39.292273 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-40d70c6b-4902-4aa9-96dc-a06571bae0d1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:39.292731 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-40d70c6b-4902-4aa9-96dc-a06571bae0d1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:39.419483 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-40d70c6b-4902-4aa9-96dc-a06571bae0d1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:39.427412 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-40d70c6b-4902-4aa9-96dc-a06571bae0d1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:39.428909 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-40d70c6b-4902-4aa9-96dc-a06571bae0d1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:39.430049 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-40d70c6b-4902-4aa9-96dc-a06571bae0d1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1597 microversion: 2.20 time: 0.150698 Jul 19 11:05:39.430602 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 101/219] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:05:39 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1597 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:05:39.430741 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-dd5b4d97-1bee-4c79-841c-9b7d8da31cfb tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:39.431207 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-dd5b4d97-1bee-4c79-841c-9b7d8da31cfb tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:39.437112 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dd5b4d97-1bee-4c79-841c-9b7d8da31cfb tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:39.437558 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dd5b4d97-1bee-4c79-841c-9b7d8da31cfb tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:39.438303 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dd5b4d97-1bee-4c79-841c-9b7d8da31cfb tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:39.438732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dd5b4d97-1bee-4c79-841c-9b7d8da31cfb tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:39.447939 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-b1296d86-df7a-4fd5-8d16-d2216a5928af tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:39.448424 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-b1296d86-df7a-4fd5-8d16-d2216a5928af tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:39.454302 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b1296d86-df7a-4fd5-8d16-d2216a5928af tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:39.454750 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b1296d86-df7a-4fd5-8d16-d2216a5928af tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:39.455503 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b1296d86-df7a-4fd5-8d16-d2216a5928af tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:39.455938 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b1296d86-df7a-4fd5-8d16-d2216a5928af tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:39.532099 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-b1296d86-df7a-4fd5-8d16-d2216a5928af tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.088298 Jul 19 11:05:39.532718 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 102/220] 10.222.0.33 () {60 vars in 1315 bytes} [Tue Jul 19 11:05:39 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326/os-interface => generated 285 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 19 11:05:39.537139 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-ce72553c-e9a6-4c9b-9486-837491a7767c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:39.537741 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-ce72553c-e9a6-4c9b-9486-837491a7767c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:39.543971 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ce72553c-e9a6-4c9b-9486-837491a7767c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:39.544418 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ce72553c-e9a6-4c9b-9486-837491a7767c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:39.545205 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ce72553c-e9a6-4c9b-9486-837491a7767c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:39.545644 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ce72553c-e9a6-4c9b-9486-837491a7767c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:39.678852 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-ce72553c-e9a6-4c9b-9486-837491a7767c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:39.687617 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-ce72553c-e9a6-4c9b-9486-837491a7767c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.153895 Jul 19 11:05:39.688193 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 103/221] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:39 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:39.732590 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-dd5b4d97-1bee-4c79-841c-9b7d8da31cfb tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:39.743527 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-dd5b4d97-1bee-4c79-841c-9b7d8da31cfb tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1313 microversion: 2.1 time: 0.316264 Jul 19 11:05:39.744530 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 119/222] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:39 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1313 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:40.187911 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-587aa317-7fea-4684-b071-71504db5dc7c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:40.188553 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-587aa317-7fea-4684-b071-71504db5dc7c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:40.194684 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-587aa317-7fea-4684-b071-71504db5dc7c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:40.195232 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-587aa317-7fea-4684-b071-71504db5dc7c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:40.196091 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-587aa317-7fea-4684-b071-71504db5dc7c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:40.196638 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-587aa317-7fea-4684-b071-71504db5dc7c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:40.337545 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-587aa317-7fea-4684-b071-71504db5dc7c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:40.345071 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-587aa317-7fea-4684-b071-71504db5dc7c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1367 microversion: 2.1 time: 0.161617 Jul 19 11:05:40.345641 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 104/223] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:40 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1367 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:40.693699 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-8e6aa965-dcc1-4ff7-a688-10c6b5f44d55 req-a8d67df0-7b11-41b1-905c-799a537f4b7d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f52fbfae-577b-403d-a4d3-9a4b79ae2326", "tag": "748a7315-79e9-4ab8-ad6a-336c057e8b2c"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:05:40.702687 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-8e6aa965-dcc1-4ff7-a688-10c6b5f44d55 req-a8d67df0-7b11-41b1-905c-799a537f4b7d service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:40.703126 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-8e6aa965-dcc1-4ff7-a688-10c6b5f44d55 req-a8d67df0-7b11-41b1-905c-799a537f4b7d service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:40.704786 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-77795858-4df7-468b-b1f5-f9c8eb8fe3ce tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:40.705627 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-77795858-4df7-468b-b1f5-f9c8eb8fe3ce tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:40.713019 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-77795858-4df7-468b-b1f5-f9c8eb8fe3ce tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:40.713701 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-77795858-4df7-468b-b1f5-f9c8eb8fe3ce tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:40.714571 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-77795858-4df7-468b-b1f5-f9c8eb8fe3ce tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:40.715176 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-77795858-4df7-468b-b1f5-f9c8eb8fe3ce tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:40.717716 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-8e6aa965-dcc1-4ff7-a688-10c6b5f44d55 req-a8d67df0-7b11-41b1-905c-799a537f4b7d service nova] Creating event network-changed:748a7315-79e9-4ab8-ad6a-336c057e8b2c for instance f52fbfae-577b-403d-a4d3-9a4b79ae2326 on hv-cinder-maste Jul 19 11:05:40.723530 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-8e6aa965-dcc1-4ff7-a688-10c6b5f44d55 req-a8d67df0-7b11-41b1-905c-799a537f4b7d service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033373 Jul 19 11:05:40.724372 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 120/224] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:05:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:05:40.762457 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-fc7fff71-8f4f-4408-bcf2-7c0cb6cb417e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:40.763150 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-fc7fff71-8f4f-4408-bcf2-7c0cb6cb417e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:40.769690 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fc7fff71-8f4f-4408-bcf2-7c0cb6cb417e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:40.770139 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fc7fff71-8f4f-4408-bcf2-7c0cb6cb417e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:40.770897 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fc7fff71-8f4f-4408-bcf2-7c0cb6cb417e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:40.771321 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fc7fff71-8f4f-4408-bcf2-7c0cb6cb417e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:40.867069 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-77795858-4df7-468b-b1f5-f9c8eb8fe3ce tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:40.875102 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-77795858-4df7-468b-b1f5-f9c8eb8fe3ce tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.174420 Jul 19 11:05:40.876305 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 105/225] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:40 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:05:40.934539 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-fc7fff71-8f4f-4408-bcf2-7c0cb6cb417e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:40.944105 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-fc7fff71-8f4f-4408-bcf2-7c0cb6cb417e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1313 microversion: 2.1 time: 0.186104 Jul 19 11:05:40.945670 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 121/226] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:40 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1313 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:05:41.362164 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-06ec8c1e-5e1b-4319-9257-ac35c9dcff7c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:41.363392 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-06ec8c1e-5e1b-4319-9257-ac35c9dcff7c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:41.370458 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-06ec8c1e-5e1b-4319-9257-ac35c9dcff7c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:41.370839 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-06ec8c1e-5e1b-4319-9257-ac35c9dcff7c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:41.371593 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-06ec8c1e-5e1b-4319-9257-ac35c9dcff7c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:41.372288 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-06ec8c1e-5e1b-4319-9257-ac35c9dcff7c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:41.503050 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-06ec8c1e-5e1b-4319-9257-ac35c9dcff7c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:41.506705 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=100 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:05:41.512901 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-06ec8c1e-5e1b-4319-9257-ac35c9dcff7c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1367 microversion: 2.1 time: 0.154506 Jul 19 11:05:41.513472 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 106/227] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:41 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1367 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:41.892458 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-49ed982b-0607-4b13-b266-c225e14ee4b0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:41.893051 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-49ed982b-0607-4b13-b266-c225e14ee4b0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:41.899432 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-49ed982b-0607-4b13-b266-c225e14ee4b0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:41.899884 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-49ed982b-0607-4b13-b266-c225e14ee4b0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:41.900689 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-49ed982b-0607-4b13-b266-c225e14ee4b0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:41.901150 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-49ed982b-0607-4b13-b266-c225e14ee4b0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:41.962666 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-dab3dc9b-7a46-4ce5-986d-e9b87d1ecb19 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:41.963182 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-dab3dc9b-7a46-4ce5-986d-e9b87d1ecb19 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:41.973667 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dab3dc9b-7a46-4ce5-986d-e9b87d1ecb19 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:41.974134 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dab3dc9b-7a46-4ce5-986d-e9b87d1ecb19 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:41.974929 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dab3dc9b-7a46-4ce5-986d-e9b87d1ecb19 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:41.977046 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dab3dc9b-7a46-4ce5-986d-e9b87d1ecb19 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:42.044223 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-49ed982b-0607-4b13-b266-c225e14ee4b0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:42.051958 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-49ed982b-0607-4b13-b266-c225e14ee4b0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.163334 Jul 19 11:05:42.052560 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 122/228] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:41 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:42.138159 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-dab3dc9b-7a46-4ce5-986d-e9b87d1ecb19 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:42.145570 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-dab3dc9b-7a46-4ce5-986d-e9b87d1ecb19 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1313 microversion: 2.1 time: 0.187811 Jul 19 11:05:42.146073 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 107/229] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:41 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1313 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:42.530105 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-5b3b44fd-04f2-497c-adb0-8d9bdf3479e7 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:42.530669 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-5b3b44fd-04f2-497c-adb0-8d9bdf3479e7 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:42.536685 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5b3b44fd-04f2-497c-adb0-8d9bdf3479e7 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:42.537243 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5b3b44fd-04f2-497c-adb0-8d9bdf3479e7 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:42.537997 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5b3b44fd-04f2-497c-adb0-8d9bdf3479e7 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:42.538436 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5b3b44fd-04f2-497c-adb0-8d9bdf3479e7 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:42.668763 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-5b3b44fd-04f2-497c-adb0-8d9bdf3479e7 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:42.676063 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-5b3b44fd-04f2-497c-adb0-8d9bdf3479e7 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1367 microversion: 2.1 time: 0.149893 Jul 19 11:05:42.676663 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 123/230] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:42 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1367 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:43.071273 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-97a55e66-d91e-4c0a-a402-faf9de47741e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:43.072199 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-97a55e66-d91e-4c0a-a402-faf9de47741e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:43.079690 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-97a55e66-d91e-4c0a-a402-faf9de47741e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:43.080144 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-97a55e66-d91e-4c0a-a402-faf9de47741e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:43.080970 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-97a55e66-d91e-4c0a-a402-faf9de47741e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:43.081411 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-97a55e66-d91e-4c0a-a402-faf9de47741e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:43.162779 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-adca904d-5999-4f06-a1fa-ca0a343a55a5 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:43.163749 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-adca904d-5999-4f06-a1fa-ca0a343a55a5 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:43.172789 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-adca904d-5999-4f06-a1fa-ca0a343a55a5 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:43.173434 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-adca904d-5999-4f06-a1fa-ca0a343a55a5 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:43.174203 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-adca904d-5999-4f06-a1fa-ca0a343a55a5 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:43.174690 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-adca904d-5999-4f06-a1fa-ca0a343a55a5 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:43.229772 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-97a55e66-d91e-4c0a-a402-faf9de47741e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:43.238845 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-97a55e66-d91e-4c0a-a402-faf9de47741e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.172808 Jul 19 11:05:43.239948 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 108/231] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:43 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:43.316907 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-adca904d-5999-4f06-a1fa-ca0a343a55a5 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:43.328980 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-adca904d-5999-4f06-a1fa-ca0a343a55a5 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1313 microversion: 2.1 time: 0.170373 Jul 19 11:05:43.329731 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 124/232] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:43 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1313 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:05:43.692359 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-296c50bc-3115-4836-8a9d-81cb76f55a09 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:43.693226 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-296c50bc-3115-4836-8a9d-81cb76f55a09 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:43.700299 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-296c50bc-3115-4836-8a9d-81cb76f55a09 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:43.700798 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-296c50bc-3115-4836-8a9d-81cb76f55a09 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:43.701573 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-296c50bc-3115-4836-8a9d-81cb76f55a09 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:43.702182 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-296c50bc-3115-4836-8a9d-81cb76f55a09 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:43.834475 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-296c50bc-3115-4836-8a9d-81cb76f55a09 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:43.841917 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-296c50bc-3115-4836-8a9d-81cb76f55a09 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1367 microversion: 2.1 time: 0.153536 Jul 19 11:05:43.842488 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 109/233] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:43 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1367 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:44.256425 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-313d2b07-0106-4d61-b82a-2997f296e556 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:44.257059 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-313d2b07-0106-4d61-b82a-2997f296e556 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:44.263025 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-313d2b07-0106-4d61-b82a-2997f296e556 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:44.263487 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-313d2b07-0106-4d61-b82a-2997f296e556 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:44.264254 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-313d2b07-0106-4d61-b82a-2997f296e556 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:44.264738 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-313d2b07-0106-4d61-b82a-2997f296e556 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:44.346875 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-4407531b-8f91-4f70-9d3b-33274f05b546 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:44.347606 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-4407531b-8f91-4f70-9d3b-33274f05b546 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:44.355749 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4407531b-8f91-4f70-9d3b-33274f05b546 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:44.356199 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4407531b-8f91-4f70-9d3b-33274f05b546 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:44.357132 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4407531b-8f91-4f70-9d3b-33274f05b546 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:44.357567 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4407531b-8f91-4f70-9d3b-33274f05b546 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:44.404912 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-313d2b07-0106-4d61-b82a-2997f296e556 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:44.413560 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-313d2b07-0106-4d61-b82a-2997f296e556 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.160923 Jul 19 11:05:44.415852 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 125/234] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:44 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:44.498768 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-4407531b-8f91-4f70-9d3b-33274f05b546 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:44.506051 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-4407531b-8f91-4f70-9d3b-33274f05b546 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1313 microversion: 2.1 time: 0.164217 Jul 19 11:05:44.506610 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 110/235] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:44 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1313 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:44.859252 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-7b246321-4b57-4f5f-bc50-c52d907c332c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:44.859859 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-7b246321-4b57-4f5f-bc50-c52d907c332c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:44.865762 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7b246321-4b57-4f5f-bc50-c52d907c332c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:44.866206 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7b246321-4b57-4f5f-bc50-c52d907c332c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:44.866952 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7b246321-4b57-4f5f-bc50-c52d907c332c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:44.867376 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7b246321-4b57-4f5f-bc50-c52d907c332c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:45.013503 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-7b246321-4b57-4f5f-bc50-c52d907c332c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:45.021107 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-7b246321-4b57-4f5f-bc50-c52d907c332c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1557 microversion: 2.1 time: 0.165658 Jul 19 11:05:45.022021 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 126/236] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:44 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1557 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:45.040228 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-1c504c4b-ce5c-4ea0-96bc-84d94d00c926 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:45.040824 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-1c504c4b-ce5c-4ea0-96bc-84d94d00c926 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:45.047481 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1c504c4b-ce5c-4ea0-96bc-84d94d00c926 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:45.048021 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1c504c4b-ce5c-4ea0-96bc-84d94d00c926 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:45.049443 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1c504c4b-ce5c-4ea0-96bc-84d94d00c926 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:45.049899 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1c504c4b-ce5c-4ea0-96bc-84d94d00c926 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:45.145801 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-1c504c4b-ce5c-4ea0-96bc-84d94d00c926 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.111422 Jul 19 11:05:45.146387 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 111/237] 10.222.0.33 () {58 vars in 1274 bytes} [Tue Jul 19 11:05:45 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372/os-interface => generated 284 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:05:45.433275 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-b299d9a9-c729-4126-a473-61d95d6a35ad tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:45.433875 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-b299d9a9-c729-4126-a473-61d95d6a35ad tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:45.444550 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b299d9a9-c729-4126-a473-61d95d6a35ad tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:45.445778 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b299d9a9-c729-4126-a473-61d95d6a35ad tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:45.446955 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b299d9a9-c729-4126-a473-61d95d6a35ad tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:45.447600 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b299d9a9-c729-4126-a473-61d95d6a35ad tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:45.523927 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-0ff67a4c-583f-429f-88b4-7fb980218226 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:45.524590 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-0ff67a4c-583f-429f-88b4-7fb980218226 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:45.533026 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0ff67a4c-583f-429f-88b4-7fb980218226 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:45.533973 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0ff67a4c-583f-429f-88b4-7fb980218226 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:45.535717 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0ff67a4c-583f-429f-88b4-7fb980218226 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:45.536604 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0ff67a4c-583f-429f-88b4-7fb980218226 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:45.902427 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b299d9a9-c729-4126-a473-61d95d6a35ad tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:45.920286 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-b299d9a9-c729-4126-a473-61d95d6a35ad tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.492114 Jul 19 11:05:45.921771 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 127/238] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:45 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 495 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:46.103748 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-0ff67a4c-583f-429f-88b4-7fb980218226 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:46.112659 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-0ff67a4c-583f-429f-88b4-7fb980218226 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1313 microversion: 2.1 time: 0.593419 Jul 19 11:05:46.113497 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 112/239] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:45 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1313 bytes in 595 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:05:46.940059 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-bf5d9647-9298-4b78-8f86-81e3ef869a5f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:46.940921 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-bf5d9647-9298-4b78-8f86-81e3ef869a5f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:46.959783 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bf5d9647-9298-4b78-8f86-81e3ef869a5f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:46.960520 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bf5d9647-9298-4b78-8f86-81e3ef869a5f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:46.962144 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bf5d9647-9298-4b78-8f86-81e3ef869a5f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:46.963198 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bf5d9647-9298-4b78-8f86-81e3ef869a5f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:47.134175 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-3281dff7-6232-475d-81ed-ae5427bc86e6 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:47.134809 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-3281dff7-6232-475d-81ed-ae5427bc86e6 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:47.142600 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3281dff7-6232-475d-81ed-ae5427bc86e6 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:47.143363 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3281dff7-6232-475d-81ed-ae5427bc86e6 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:47.144332 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3281dff7-6232-475d-81ed-ae5427bc86e6 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:47.145054 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3281dff7-6232-475d-81ed-ae5427bc86e6 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:47.265805 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-bf5d9647-9298-4b78-8f86-81e3ef869a5f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:47.274470 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-bf5d9647-9298-4b78-8f86-81e3ef869a5f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.340569 Jul 19 11:05:47.286718 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 128/240] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:46 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:47.403415 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-d787a41b-1559-40e4-8341-3f636ffc923d req-b5c3deac-5248-48b2-819f-f4845c507ec4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4c71f564-de63-4fcd-88ce-01ecba3d8372", "tag": "526a30d7-5fcf-4c45-86bf-6ddbc902d48c"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:05:47.411547 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-d787a41b-1559-40e4-8341-3f636ffc923d req-b5c3deac-5248-48b2-819f-f4845c507ec4 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:47.412307 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-d787a41b-1559-40e4-8341-3f636ffc923d req-b5c3deac-5248-48b2-819f-f4845c507ec4 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:47.433625 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-d787a41b-1559-40e4-8341-3f636ffc923d req-b5c3deac-5248-48b2-819f-f4845c507ec4 service nova] Creating event network-changed:526a30d7-5fcf-4c45-86bf-6ddbc902d48c for instance 4c71f564-de63-4fcd-88ce-01ecba3d8372 on hv-cinder-maste Jul 19 11:05:47.440660 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-d787a41b-1559-40e4-8341-3f636ffc923d req-b5c3deac-5248-48b2-819f-f4845c507ec4 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040729 Jul 19 11:05:47.441550 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 129/241] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:05:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:05:47.471578 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-3281dff7-6232-475d-81ed-ae5427bc86e6 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:47.482195 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-3281dff7-6232-475d-81ed-ae5427bc86e6 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1313 microversion: 2.1 time: 0.354067 Jul 19 11:05:47.482828 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 113/242] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:47 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1313 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:48.291973 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-00f12ff8-d50a-4858-b7a5-a43fc0497be4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:48.293372 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-00f12ff8-d50a-4858-b7a5-a43fc0497be4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:48.300199 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-00f12ff8-d50a-4858-b7a5-a43fc0497be4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:48.300649 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-00f12ff8-d50a-4858-b7a5-a43fc0497be4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:48.304538 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-00f12ff8-d50a-4858-b7a5-a43fc0497be4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:48.305015 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-00f12ff8-d50a-4858-b7a5-a43fc0497be4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:48.499584 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-f72e7e79-0552-4cb1-b456-3326f2b04560 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:48.500193 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-f72e7e79-0552-4cb1-b456-3326f2b04560 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:48.522679 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f72e7e79-0552-4cb1-b456-3326f2b04560 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:48.523322 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f72e7e79-0552-4cb1-b456-3326f2b04560 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:48.524398 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f72e7e79-0552-4cb1-b456-3326f2b04560 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:48.526180 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f72e7e79-0552-4cb1-b456-3326f2b04560 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:48.535080 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-00f12ff8-d50a-4858-b7a5-a43fc0497be4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:48.544776 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-00f12ff8-d50a-4858-b7a5-a43fc0497be4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.256865 Jul 19 11:05:48.548651 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 130/243] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:48 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:49.023334 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-f72e7e79-0552-4cb1-b456-3326f2b04560 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:49.033315 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-f72e7e79-0552-4cb1-b456-3326f2b04560 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1313 microversion: 2.1 time: 0.538026 Jul 19 11:05:49.036761 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 114/244] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:48 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1313 bytes in 542 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:49.564234 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-204d63e8-732b-489b-9433-1044d850cce0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:49.564880 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-204d63e8-732b-489b-9433-1044d850cce0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:49.573311 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-204d63e8-732b-489b-9433-1044d850cce0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:49.574011 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-204d63e8-732b-489b-9433-1044d850cce0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:49.574986 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-204d63e8-732b-489b-9433-1044d850cce0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:49.575614 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-204d63e8-732b-489b-9433-1044d850cce0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:49.743468 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-204d63e8-732b-489b-9433-1044d850cce0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:49.746377 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:05:49.756257 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-204d63e8-732b-489b-9433-1044d850cce0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.195963 Jul 19 11:05:49.757184 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 131/245] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:49 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:50.055022 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-1466d9dd-072f-4ba2-b9ba-d6666dbd915e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:50.055022 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-1466d9dd-072f-4ba2-b9ba-d6666dbd915e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:50.060718 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1466d9dd-072f-4ba2-b9ba-d6666dbd915e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:50.061176 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1466d9dd-072f-4ba2-b9ba-d6666dbd915e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:50.061937 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1466d9dd-072f-4ba2-b9ba-d6666dbd915e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:50.062372 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1466d9dd-072f-4ba2-b9ba-d6666dbd915e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:50.244656 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-1466d9dd-072f-4ba2-b9ba-d6666dbd915e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:50.254886 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-1466d9dd-072f-4ba2-b9ba-d6666dbd915e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1313 microversion: 2.1 time: 0.205787 Jul 19 11:05:50.259834 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 115/246] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:50 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1313 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:50.774350 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-fd251b1e-70b4-41bc-996f-1b64792f6b9e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:50.774957 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-fd251b1e-70b4-41bc-996f-1b64792f6b9e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:50.782555 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fd251b1e-70b4-41bc-996f-1b64792f6b9e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:50.783005 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fd251b1e-70b4-41bc-996f-1b64792f6b9e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:50.783782 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fd251b1e-70b4-41bc-996f-1b64792f6b9e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:50.784213 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fd251b1e-70b4-41bc-996f-1b64792f6b9e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:50.974354 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-fd251b1e-70b4-41bc-996f-1b64792f6b9e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:50.983031 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-fd251b1e-70b4-41bc-996f-1b64792f6b9e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.212891 Jul 19 11:05:50.983610 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 132/247] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:50 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:51.274047 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-5621d31b-f51c-46aa-b3c3-897bfe26fc8d tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:51.277052 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-5621d31b-f51c-46aa-b3c3-897bfe26fc8d tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:51.286706 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5621d31b-f51c-46aa-b3c3-897bfe26fc8d tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:51.287366 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5621d31b-f51c-46aa-b3c3-897bfe26fc8d tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:51.288371 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5621d31b-f51c-46aa-b3c3-897bfe26fc8d tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:51.288859 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5621d31b-f51c-46aa-b3c3-897bfe26fc8d tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:51.497567 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-5621d31b-f51c-46aa-b3c3-897bfe26fc8d tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:51.505959 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-5621d31b-f51c-46aa-b3c3-897bfe26fc8d tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1313 microversion: 2.1 time: 0.238137 Jul 19 11:05:51.506714 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 116/248] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:51 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1313 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:52.000956 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-adcf8bb0-5e47-4f1c-89bc-5bc35b233cb4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:52.001558 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-adcf8bb0-5e47-4f1c-89bc-5bc35b233cb4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:52.008460 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-adcf8bb0-5e47-4f1c-89bc-5bc35b233cb4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:52.008971 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-adcf8bb0-5e47-4f1c-89bc-5bc35b233cb4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:52.009906 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-adcf8bb0-5e47-4f1c-89bc-5bc35b233cb4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:52.010342 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-adcf8bb0-5e47-4f1c-89bc-5bc35b233cb4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:52.164944 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-adcf8bb0-5e47-4f1c-89bc-5bc35b233cb4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:52.174211 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-adcf8bb0-5e47-4f1c-89bc-5bc35b233cb4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.177408 Jul 19 11:05:52.174990 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 133/249] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:51 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:52.280981 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-71970a19-64df-441f-b1c3-5fa2ebf68372 req-c84d6fbf-2025-4f7b-87cd-ccca87f3b7be service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b4e0099f-88fa-4929-94a2-ef54325bace7", "name": "network-vif-plugged", "status": "completed", "tag": "76b1d057-9d32-40d9-8cd9-1ba190a6841d"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:05:52.289945 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-71970a19-64df-441f-b1c3-5fa2ebf68372 req-c84d6fbf-2025-4f7b-87cd-ccca87f3b7be service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:52.290394 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-71970a19-64df-441f-b1c3-5fa2ebf68372 req-c84d6fbf-2025-4f7b-87cd-ccca87f3b7be service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:52.309381 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-71970a19-64df-441f-b1c3-5fa2ebf68372 req-c84d6fbf-2025-4f7b-87cd-ccca87f3b7be service nova] Creating event network-vif-plugged:76b1d057-9d32-40d9-8cd9-1ba190a6841d for instance b4e0099f-88fa-4929-94a2-ef54325bace7 on hv-cinder-maste Jul 19 11:05:52.314450 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-71970a19-64df-441f-b1c3-5fa2ebf68372 req-c84d6fbf-2025-4f7b-87cd-ccca87f3b7be service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037672 Jul 19 11:05:52.315213 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 117/250] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:05:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:05:52.523491 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-12bb4cef-aa21-4ccc-9203-80e9997d6fb4 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:52.524452 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-12bb4cef-aa21-4ccc-9203-80e9997d6fb4 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:52.531145 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-12bb4cef-aa21-4ccc-9203-80e9997d6fb4 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:52.531822 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-12bb4cef-aa21-4ccc-9203-80e9997d6fb4 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:52.533368 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-12bb4cef-aa21-4ccc-9203-80e9997d6fb4 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:52.534197 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-12bb4cef-aa21-4ccc-9203-80e9997d6fb4 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:52.753624 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-12bb4cef-aa21-4ccc-9203-80e9997d6fb4 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:52.762826 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-12bb4cef-aa21-4ccc-9203-80e9997d6fb4 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1313 microversion: 2.1 time: 0.243690 Jul 19 11:05:52.764022 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 134/251] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:52 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1313 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:53.194297 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-1efaa23b-a8ed-4777-aa9a-3900fc76b0dd tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:53.195000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-1efaa23b-a8ed-4777-aa9a-3900fc76b0dd tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:53.216629 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1efaa23b-a8ed-4777-aa9a-3900fc76b0dd tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:53.217283 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1efaa23b-a8ed-4777-aa9a-3900fc76b0dd tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:53.218211 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1efaa23b-a8ed-4777-aa9a-3900fc76b0dd tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:53.219962 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1efaa23b-a8ed-4777-aa9a-3900fc76b0dd tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:53.447820 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-1efaa23b-a8ed-4777-aa9a-3900fc76b0dd tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:53.456352 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-1efaa23b-a8ed-4777-aa9a-3900fc76b0dd tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.266959 Jul 19 11:05:53.457267 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 118/252] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:53 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:53.474475 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-962e1e43-d30f-476d-9c53-648eb15d991b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:53.475030 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-962e1e43-d30f-476d-9c53-648eb15d991b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:53.483728 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-962e1e43-d30f-476d-9c53-648eb15d991b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:53.484383 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-962e1e43-d30f-476d-9c53-648eb15d991b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:53.485543 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-962e1e43-d30f-476d-9c53-648eb15d991b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:53.486536 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-962e1e43-d30f-476d-9c53-648eb15d991b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:53.516243 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-962e1e43-d30f-476d-9c53-648eb15d991b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Going to try to terminate instance {{(pid=117849) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 19 11:05:53.607139 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-962e1e43-d30f-476d-9c53-648eb15d991b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "DELETE /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 204 len: 0 microversion: 2.1 time: 0.137194 Jul 19 11:05:53.607728 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 135/253] 10.222.0.33 () {58 vars in 1238 bytes} [Tue Jul 19 11:05:53 2022] DELETE /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 19 11:05:53.615628 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-f6b34162-b181-42a7-a86e-f5a9d1590ab5 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:53.616349 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-f6b34162-b181-42a7-a86e-f5a9d1590ab5 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:53.623444 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f6b34162-b181-42a7-a86e-f5a9d1590ab5 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:53.623962 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f6b34162-b181-42a7-a86e-f5a9d1590ab5 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:53.624836 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f6b34162-b181-42a7-a86e-f5a9d1590ab5 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:53.625342 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f6b34162-b181-42a7-a86e-f5a9d1590ab5 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:53.755723 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-f6b34162-b181-42a7-a86e-f5a9d1590ab5 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:53.765451 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-f6b34162-b181-42a7-a86e-f5a9d1590ab5 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1554 microversion: 2.1 time: 0.153225 Jul 19 11:05:53.766885 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 119/254] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:53 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1554 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:53.781536 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-ae2c2eff-41e2-4502-ac98-e8d269169239 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:53.782233 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-ae2c2eff-41e2-4502-ac98-e8d269169239 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:53.784233 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:05:53.790817 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ae2c2eff-41e2-4502-ac98-e8d269169239 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:53.791340 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ae2c2eff-41e2-4502-ac98-e8d269169239 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:53.792124 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ae2c2eff-41e2-4502-ac98-e8d269169239 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:53.792560 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ae2c2eff-41e2-4502-ac98-e8d269169239 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:53.921394 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-ae2c2eff-41e2-4502-ac98-e8d269169239 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:53.928927 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-ae2c2eff-41e2-4502-ac98-e8d269169239 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1313 microversion: 2.1 time: 0.151646 Jul 19 11:05:53.930021 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 136/255] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:53 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1313 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:54.474063 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-60ae3684-5dbf-49a3-94cf-0104a077b3fd tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:54.474720 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-60ae3684-5dbf-49a3-94cf-0104a077b3fd tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:54.482069 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-60ae3684-5dbf-49a3-94cf-0104a077b3fd tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:54.482514 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-60ae3684-5dbf-49a3-94cf-0104a077b3fd tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:54.483332 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-60ae3684-5dbf-49a3-94cf-0104a077b3fd tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:54.483796 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-60ae3684-5dbf-49a3-94cf-0104a077b3fd tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:54.614796 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-60ae3684-5dbf-49a3-94cf-0104a077b3fd tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:54.623442 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-60ae3684-5dbf-49a3-94cf-0104a077b3fd tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.153311 Jul 19 11:05:54.624045 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 120/256] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:54 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:54.782976 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-8aac68f8-2c3a-4e88-87bf-a9c077027b4d tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:54.783578 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-8aac68f8-2c3a-4e88-87bf-a9c077027b4d tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:54.789611 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8aac68f8-2c3a-4e88-87bf-a9c077027b4d tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:54.790054 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8aac68f8-2c3a-4e88-87bf-a9c077027b4d tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:54.790820 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8aac68f8-2c3a-4e88-87bf-a9c077027b4d tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:54.791260 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8aac68f8-2c3a-4e88-87bf-a9c077027b4d tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:54.928266 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-8aac68f8-2c3a-4e88-87bf-a9c077027b4d tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:54.936664 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-8aac68f8-2c3a-4e88-87bf-a9c077027b4d tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1554 microversion: 2.1 time: 0.157466 Jul 19 11:05:54.937827 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 137/257] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:54 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1554 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:54.946091 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-ca996d67-8286-4063-9340-4c06cae5e885 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:54.946799 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-ca996d67-8286-4063-9340-4c06cae5e885 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:54.952811 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ca996d67-8286-4063-9340-4c06cae5e885 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:54.953261 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ca996d67-8286-4063-9340-4c06cae5e885 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:54.954011 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ca996d67-8286-4063-9340-4c06cae5e885 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:54.954444 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ca996d67-8286-4063-9340-4c06cae5e885 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:55.081181 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-ca996d67-8286-4063-9340-4c06cae5e885 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:55.089073 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-ca996d67-8286-4063-9340-4c06cae5e885 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1313 microversion: 2.1 time: 0.146740 Jul 19 11:05:55.089639 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 121/258] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:54 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1313 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:55.640055 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-00030d8f-1776-4741-8a25-370abf607c73 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:55.640806 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-00030d8f-1776-4741-8a25-370abf607c73 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:55.646715 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-00030d8f-1776-4741-8a25-370abf607c73 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:55.647159 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-00030d8f-1776-4741-8a25-370abf607c73 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:55.647913 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-00030d8f-1776-4741-8a25-370abf607c73 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:55.648348 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-00030d8f-1776-4741-8a25-370abf607c73 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:55.780679 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-00030d8f-1776-4741-8a25-370abf607c73 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:55.791670 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-00030d8f-1776-4741-8a25-370abf607c73 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.155290 Jul 19 11:05:55.792509 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 138/259] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:55 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:55.954262 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-30288c97-8b2c-44aa-bae1-0393790f1792 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:55.954869 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-30288c97-8b2c-44aa-bae1-0393790f1792 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:55.957259 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_api:SELECT=79 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:05:55.962710 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-30288c97-8b2c-44aa-bae1-0393790f1792 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:55.963159 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-30288c97-8b2c-44aa-bae1-0393790f1792 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:55.963918 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-30288c97-8b2c-44aa-bae1-0393790f1792 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:55.964486 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-30288c97-8b2c-44aa-bae1-0393790f1792 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:56.103768 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-30288c97-8b2c-44aa-bae1-0393790f1792 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:56.109064 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-c524dcd3-ca0d-42f7-b432-e29b4a5f3346 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:56.109833 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-c524dcd3-ca0d-42f7-b432-e29b4a5f3346 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:56.113735 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-30288c97-8b2c-44aa-bae1-0393790f1792 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1554 microversion: 2.1 time: 0.163352 Jul 19 11:05:56.114452 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 122/260] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:55 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1554 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:56.117943 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c524dcd3-ca0d-42f7-b432-e29b4a5f3346 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:56.118509 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c524dcd3-ca0d-42f7-b432-e29b4a5f3346 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:56.119625 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c524dcd3-ca0d-42f7-b432-e29b4a5f3346 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:56.120267 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c524dcd3-ca0d-42f7-b432-e29b4a5f3346 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:56.302609 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-c524dcd3-ca0d-42f7-b432-e29b4a5f3346 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:56.312314 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-c524dcd3-ca0d-42f7-b432-e29b4a5f3346 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1313 microversion: 2.1 time: 0.209051 Jul 19 11:05:56.317813 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 139/261] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:56 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1313 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:56.811395 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-10593422-4f7d-4b7a-8824-30ac497ea167 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:56.812087 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-10593422-4f7d-4b7a-8824-30ac497ea167 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:56.822156 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-10593422-4f7d-4b7a-8824-30ac497ea167 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:56.822605 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-10593422-4f7d-4b7a-8824-30ac497ea167 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:56.823507 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-10593422-4f7d-4b7a-8824-30ac497ea167 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:56.823797 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-10593422-4f7d-4b7a-8824-30ac497ea167 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:56.984384 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-10593422-4f7d-4b7a-8824-30ac497ea167 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:56.995684 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-10593422-4f7d-4b7a-8824-30ac497ea167 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.189376 Jul 19 11:05:56.998572 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 123/262] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:56 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:57.131665 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-c5962539-09d8-4eab-82bf-df80935dd6c7 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:57.132383 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-c5962539-09d8-4eab-82bf-df80935dd6c7 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:57.140246 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c5962539-09d8-4eab-82bf-df80935dd6c7 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:57.140903 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c5962539-09d8-4eab-82bf-df80935dd6c7 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:57.141849 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c5962539-09d8-4eab-82bf-df80935dd6c7 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:57.142516 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c5962539-09d8-4eab-82bf-df80935dd6c7 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:57.294764 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-c5962539-09d8-4eab-82bf-df80935dd6c7 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:57.305796 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-c5962539-09d8-4eab-82bf-df80935dd6c7 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1554 microversion: 2.1 time: 0.178062 Jul 19 11:05:57.306371 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 140/263] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:57 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1554 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:57.329753 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-2099d260-3d37-4da0-bf55-582b311e7c5f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:57.330260 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-2099d260-3d37-4da0-bf55-582b311e7c5f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:57.336984 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2099d260-3d37-4da0-bf55-582b311e7c5f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:57.337434 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2099d260-3d37-4da0-bf55-582b311e7c5f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:57.338224 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2099d260-3d37-4da0-bf55-582b311e7c5f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:57.338657 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2099d260-3d37-4da0-bf55-582b311e7c5f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:57.359528 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=100 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:05:57.543337 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-2099d260-3d37-4da0-bf55-582b311e7c5f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:57.552056 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-2099d260-3d37-4da0-bf55-582b311e7c5f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1498 microversion: 2.1 time: 0.226460 Jul 19 11:05:57.565179 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 124/264] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:57 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1498 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:05:58.017191 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-71373548-4e06-46b6-9cd8-17284ced312f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:58.018401 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-71373548-4e06-46b6-9cd8-17284ced312f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:58.029700 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-71373548-4e06-46b6-9cd8-17284ced312f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:58.030332 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-71373548-4e06-46b6-9cd8-17284ced312f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:58.031122 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-71373548-4e06-46b6-9cd8-17284ced312f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:58.032498 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-71373548-4e06-46b6-9cd8-17284ced312f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:58.292571 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-71373548-4e06-46b6-9cd8-17284ced312f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:58.300374 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-71373548-4e06-46b6-9cd8-17284ced312f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.289069 Jul 19 11:05:58.303169 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 141/265] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:58 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:58.330572 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-d50f70a2-9c3e-4bed-ab11-af7d4cd6b09b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:58.331950 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-d50f70a2-9c3e-4bed-ab11-af7d4cd6b09b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:58.344643 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d50f70a2-9c3e-4bed-ab11-af7d4cd6b09b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:58.348321 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d50f70a2-9c3e-4bed-ab11-af7d4cd6b09b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:58.350394 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d50f70a2-9c3e-4bed-ab11-af7d4cd6b09b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:58.350914 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d50f70a2-9c3e-4bed-ab11-af7d4cd6b09b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:58.428357 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-7b9e6072-825a-404a-990c-c369a29a525f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:58.429392 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-7b9e6072-825a-404a-990c-c369a29a525f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:58.436639 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7b9e6072-825a-404a-990c-c369a29a525f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:58.437344 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7b9e6072-825a-404a-990c-c369a29a525f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:58.438720 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7b9e6072-825a-404a-990c-c369a29a525f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:58.439855 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7b9e6072-825a-404a-990c-c369a29a525f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:58.633049 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-7b9e6072-825a-404a-990c-c369a29a525f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Going to try to terminate instance {{(pid=117849) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 19 11:05:58.738456 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-7b9e6072-825a-404a-990c-c369a29a525f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "DELETE /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 204 len: 0 microversion: 2.1 time: 0.312469 Jul 19 11:05:58.739113 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 142/266] 10.222.0.33 () {58 vars in 1238 bytes} [Tue Jul 19 11:05:58 2022] DELETE /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 19 11:05:58.743383 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-d50f70a2-9c3e-4bed-ab11-af7d4cd6b09b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:58.750186 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-754eb4b6-e29b-4895-8d6e-76211950783a tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:58.751568 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-754eb4b6-e29b-4895-8d6e-76211950783a tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:58.754955 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-d50f70a2-9c3e-4bed-ab11-af7d4cd6b09b tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1554 microversion: 2.1 time: 0.429231 Jul 19 11:05:58.756070 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 125/267] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:58 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1554 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:58.760485 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-754eb4b6-e29b-4895-8d6e-76211950783a tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:58.761168 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-754eb4b6-e29b-4895-8d6e-76211950783a tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:58.762875 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-754eb4b6-e29b-4895-8d6e-76211950783a tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:58.763469 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-754eb4b6-e29b-4895-8d6e-76211950783a tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:59.017470 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-754eb4b6-e29b-4895-8d6e-76211950783a tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:59.028769 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-754eb4b6-e29b-4895-8d6e-76211950783a tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1504 microversion: 2.1 time: 0.281869 Jul 19 11:05:59.032144 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 143/268] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:58 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1504 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:59.225097 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-127f70e9-2340-41a5-ac40-1d35bd4acedf req-b1ec8c7a-b137-452b-9351-eaec4c990685 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "126c2860-81b1-42fc-8fbd-78178837f2c2", "name": "network-vif-deleted", "tag": "ab2b6732-5d45-4b40-8888-44a729c6870f"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:05:59.235660 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-127f70e9-2340-41a5-ac40-1d35bd4acedf req-b1ec8c7a-b137-452b-9351-eaec4c990685 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:59.236527 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-127f70e9-2340-41a5-ac40-1d35bd4acedf req-b1ec8c7a-b137-452b-9351-eaec4c990685 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:59.250568 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-127f70e9-2340-41a5-ac40-1d35bd4acedf req-b1ec8c7a-b137-452b-9351-eaec4c990685 service nova] Creating event network-vif-deleted:ab2b6732-5d45-4b40-8888-44a729c6870f for instance 126c2860-81b1-42fc-8fbd-78178837f2c2 on hv-cinder-maste Jul 19 11:05:59.257186 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-127f70e9-2340-41a5-ac40-1d35bd4acedf req-b1ec8c7a-b137-452b-9351-eaec4c990685 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036359 Jul 19 11:05:59.258314 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 126/269] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:05:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:05:59.323876 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-acbd0e08-bef0-4ce1-91c4-0e7fc1e7b83f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:59.324489 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-acbd0e08-bef0-4ce1-91c4-0e7fc1e7b83f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:59.331606 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-acbd0e08-bef0-4ce1-91c4-0e7fc1e7b83f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:59.332437 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-acbd0e08-bef0-4ce1-91c4-0e7fc1e7b83f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:59.333498 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-acbd0e08-bef0-4ce1-91c4-0e7fc1e7b83f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:59.334138 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-acbd0e08-bef0-4ce1-91c4-0e7fc1e7b83f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:59.674248 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-acbd0e08-bef0-4ce1-91c4-0e7fc1e7b83f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:59.682334 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-acbd0e08-bef0-4ce1-91c4-0e7fc1e7b83f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.366671 Jul 19 11:05:59.683490 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 144/270] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:59 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:05:59.774433 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-827011fd-c722-4957-a27e-3c30f332480c tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:05:59.775338 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-827011fd-c722-4957-a27e-3c30f332480c tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:05:59.781831 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-827011fd-c722-4957-a27e-3c30f332480c tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:59.782355 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-827011fd-c722-4957-a27e-3c30f332480c tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:59.783217 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-827011fd-c722-4957-a27e-3c30f332480c tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:05:59.783855 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-827011fd-c722-4957-a27e-3c30f332480c tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:05:59.857122 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-827011fd-c722-4957-a27e-3c30f332480c tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:05:59.867906 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-827011fd-c722-4957-a27e-3c30f332480c tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1512 microversion: 2.1 time: 0.098917 Jul 19 11:05:59.868670 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 127/271] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:05:59 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1512 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:00.053386 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-6d7c0c0d-c8f2-4d0e-9158-efea3a998410 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:00.054895 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-6d7c0c0d-c8f2-4d0e-9158-efea3a998410 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:00.063488 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6d7c0c0d-c8f2-4d0e-9158-efea3a998410 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:00.064300 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6d7c0c0d-c8f2-4d0e-9158-efea3a998410 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:00.065343 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6d7c0c0d-c8f2-4d0e-9158-efea3a998410 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:00.066016 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6d7c0c0d-c8f2-4d0e-9158-efea3a998410 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:00.071991 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:06:00.298185 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-6d7c0c0d-c8f2-4d0e-9158-efea3a998410 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:00.322553 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-6d7c0c0d-c8f2-4d0e-9158-efea3a998410 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1504 microversion: 2.1 time: 0.272429 Jul 19 11:06:00.323497 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 145/272] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:00 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1504 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:00.709634 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-36a045ce-5740-4c6a-b6eb-c59e814569ee tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:00.711205 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-36a045ce-5740-4c6a-b6eb-c59e814569ee tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:00.723545 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-36a045ce-5740-4c6a-b6eb-c59e814569ee tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:00.724213 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-36a045ce-5740-4c6a-b6eb-c59e814569ee tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:00.725691 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-36a045ce-5740-4c6a-b6eb-c59e814569ee tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:00.726867 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-36a045ce-5740-4c6a-b6eb-c59e814569ee tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:00.886705 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-326e0e2b-6f02-4303-9ddb-3db01c9fd901 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:00.888082 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-326e0e2b-6f02-4303-9ddb-3db01c9fd901 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:00.895072 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-326e0e2b-6f02-4303-9ddb-3db01c9fd901 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:00.896234 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-326e0e2b-6f02-4303-9ddb-3db01c9fd901 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:00.897292 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-326e0e2b-6f02-4303-9ddb-3db01c9fd901 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:00.898711 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-326e0e2b-6f02-4303-9ddb-3db01c9fd901 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:01.022287 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-326e0e2b-6f02-4303-9ddb-3db01c9fd901 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:01.042083 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-326e0e2b-6f02-4303-9ddb-3db01c9fd901 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1512 microversion: 2.1 time: 0.160288 Jul 19 11:06:01.042759 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 146/273] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:00 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1512 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:01.054490 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-36a045ce-5740-4c6a-b6eb-c59e814569ee tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:01.069157 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-36a045ce-5740-4c6a-b6eb-c59e814569ee tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.372003 Jul 19 11:06:01.072882 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 128/274] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:00 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:01.339655 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-ea95e5d1-1b54-4ccb-b32c-6d1795ddb596 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:01.340375 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-ea95e5d1-1b54-4ccb-b32c-6d1795ddb596 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:01.353278 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ea95e5d1-1b54-4ccb-b32c-6d1795ddb596 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:01.353826 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ea95e5d1-1b54-4ccb-b32c-6d1795ddb596 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:01.354682 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ea95e5d1-1b54-4ccb-b32c-6d1795ddb596 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:01.355227 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ea95e5d1-1b54-4ccb-b32c-6d1795ddb596 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:01.525834 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-ea95e5d1-1b54-4ccb-b32c-6d1795ddb596 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:01.534018 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-ea95e5d1-1b54-4ccb-b32c-6d1795ddb596 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1504 microversion: 2.1 time: 0.198405 Jul 19 11:06:01.534936 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 147/275] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:01 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1504 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:02.059895 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-f5d68ae3-edbe-4bc8-b241-a5b0ba4f8e92 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:02.060506 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-f5d68ae3-edbe-4bc8-b241-a5b0ba4f8e92 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:02.078508 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f5d68ae3-edbe-4bc8-b241-a5b0ba4f8e92 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:02.079005 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f5d68ae3-edbe-4bc8-b241-a5b0ba4f8e92 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:02.079892 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f5d68ae3-edbe-4bc8-b241-a5b0ba4f8e92 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:02.080338 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f5d68ae3-edbe-4bc8-b241-a5b0ba4f8e92 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:02.089738 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-00c38baa-f41a-4629-81bd-789fd25b645b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:02.090886 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-00c38baa-f41a-4629-81bd-789fd25b645b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:02.108031 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-00c38baa-f41a-4629-81bd-789fd25b645b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:02.108492 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-00c38baa-f41a-4629-81bd-789fd25b645b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:02.112483 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-00c38baa-f41a-4629-81bd-789fd25b645b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:02.113307 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-00c38baa-f41a-4629-81bd-789fd25b645b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:02.272857 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-f5d68ae3-edbe-4bc8-b241-a5b0ba4f8e92 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:02.286972 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-f5d68ae3-edbe-4bc8-b241-a5b0ba4f8e92 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1512 microversion: 2.1 time: 0.230986 Jul 19 11:06:02.291971 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 129/276] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:02 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1512 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:02.352246 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-00c38baa-f41a-4629-81bd-789fd25b645b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:02.361291 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-00c38baa-f41a-4629-81bd-789fd25b645b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.275844 Jul 19 11:06:02.365075 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 148/277] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:02 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:06:02.551724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-f16b9a01-c573-46a5-ba24-dedc21dc389b tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:02.552657 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-f16b9a01-c573-46a5-ba24-dedc21dc389b tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:02.560354 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f16b9a01-c573-46a5-ba24-dedc21dc389b tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:02.561086 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f16b9a01-c573-46a5-ba24-dedc21dc389b tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:02.561877 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f16b9a01-c573-46a5-ba24-dedc21dc389b tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:02.562582 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f16b9a01-c573-46a5-ba24-dedc21dc389b tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:02.974169 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-f16b9a01-c573-46a5-ba24-dedc21dc389b tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:02.990304 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-f16b9a01-c573-46a5-ba24-dedc21dc389b tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1504 microversion: 2.1 time: 0.442819 Jul 19 11:06:03.007751 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 130/278] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:02 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1504 bytes in 461 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:03.308598 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-8957d1fc-2c21-4077-be5c-d29421ccec60 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:03.310031 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-8957d1fc-2c21-4077-be5c-d29421ccec60 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:03.320497 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8957d1fc-2c21-4077-be5c-d29421ccec60 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:03.321149 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8957d1fc-2c21-4077-be5c-d29421ccec60 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:03.324650 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8957d1fc-2c21-4077-be5c-d29421ccec60 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:03.325280 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8957d1fc-2c21-4077-be5c-d29421ccec60 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:03.379162 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-4b45f1c3-2b61-4b34-8828-a19492b4756c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:03.380359 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-4b45f1c3-2b61-4b34-8828-a19492b4756c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:03.385692 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8957d1fc-2c21-4077-be5c-d29421ccec60 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:03.386428 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8957d1fc-2c21-4077-be5c-d29421ccec60 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:03.387272 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.objects.instance [None req-8957d1fc-2c21-4077-be5c-d29421ccec60 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lazy-loading 'fault' on Instance uuid 126c2860-81b1-42fc-8fbd-78178837f2c2 {{(pid=117849) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 19 11:06:03.390738 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4b45f1c3-2b61-4b34-8828-a19492b4756c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:03.391869 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4b45f1c3-2b61-4b34-8828-a19492b4756c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:03.393966 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4b45f1c3-2b61-4b34-8828-a19492b4756c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:03.396584 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4b45f1c3-2b61-4b34-8828-a19492b4756c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:03.486473 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-8957d1fc-2c21-4077-be5c-d29421ccec60 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6cb55e4c41f4e6d94156c597a1baad8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '205683784b714ab399311aeca15d0cfa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:03.499806 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-8957d1fc-2c21-4077-be5c-d29421ccec60 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 200 len: 1350 microversion: 2.1 time: 0.194658 Jul 19 11:06:03.501274 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 149/279] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:03 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 1350 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:03.751485 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-4b45f1c3-2b61-4b34-8828-a19492b4756c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:03.756979 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-5b84cef9-8cdc-4e71-8a76-3cca70bbfe57 req-110f5616-393d-4373-bef7-cc3979f0c070 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b4e0099f-88fa-4929-94a2-ef54325bace7", "name": "network-vif-deleted", "tag": "76b1d057-9d32-40d9-8cd9-1ba190a6841d"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:06:03.773191 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-4b45f1c3-2b61-4b34-8828-a19492b4756c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.397037 Jul 19 11:06:03.775268 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 131/280] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:03 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:03.779309 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-5b84cef9-8cdc-4e71-8a76-3cca70bbfe57 req-110f5616-393d-4373-bef7-cc3979f0c070 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:03.779757 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-5b84cef9-8cdc-4e71-8a76-3cca70bbfe57 req-110f5616-393d-4373-bef7-cc3979f0c070 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:03.812153 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-5b84cef9-8cdc-4e71-8a76-3cca70bbfe57 req-110f5616-393d-4373-bef7-cc3979f0c070 service nova] Creating event network-vif-deleted:76b1d057-9d32-40d9-8cd9-1ba190a6841d for instance b4e0099f-88fa-4929-94a2-ef54325bace7 on hv-cinder-maste Jul 19 11:06:03.827701 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-5b84cef9-8cdc-4e71-8a76-3cca70bbfe57 req-110f5616-393d-4373-bef7-cc3979f0c070 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.077767 Jul 19 11:06:03.831467 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 150/281] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:06:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:06:04.010338 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-5977a93b-ebf7-4b90-979c-8f7428852b7f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:04.011740 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-5977a93b-ebf7-4b90-979c-8f7428852b7f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:04.024231 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5977a93b-ebf7-4b90-979c-8f7428852b7f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:04.025303 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5977a93b-ebf7-4b90-979c-8f7428852b7f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:04.026556 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5977a93b-ebf7-4b90-979c-8f7428852b7f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:04.027293 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5977a93b-ebf7-4b90-979c-8f7428852b7f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:04.082768 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5977a93b-ebf7-4b90-979c-8f7428852b7f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:04.083220 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5977a93b-ebf7-4b90-979c-8f7428852b7f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:04.083687 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.objects.instance [None req-5977a93b-ebf7-4b90-979c-8f7428852b7f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lazy-loading 'fault' on Instance uuid b4e0099f-88fa-4929-94a2-ef54325bace7 {{(pid=117848) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 19 11:06:04.159706 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-5977a93b-ebf7-4b90-979c-8f7428852b7f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:04.168340 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-5977a93b-ebf7-4b90-979c-8f7428852b7f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 200 len: 1303 microversion: 2.1 time: 0.161614 Jul 19 11:06:04.168557 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 132/282] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:04 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 1303 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:04.519423 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-4fb0f2e6-ac5a-402f-a75e-9f701f45b8bf tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:04.520215 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-4fb0f2e6-ac5a-402f-a75e-9f701f45b8bf tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:04.534409 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4fb0f2e6-ac5a-402f-a75e-9f701f45b8bf tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:04.534524 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4fb0f2e6-ac5a-402f-a75e-9f701f45b8bf tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:04.535236 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4fb0f2e6-ac5a-402f-a75e-9f701f45b8bf tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:04.535839 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4fb0f2e6-ac5a-402f-a75e-9f701f45b8bf tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:04.572117 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.wsgi [None req-4fb0f2e6-ac5a-402f-a75e-9f701f45b8bf tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] HTTP exception thrown: Instance 126c2860-81b1-42fc-8fbd-78178837f2c2 could not be found. Jul 19 11:06:04.576722 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-4fb0f2e6-ac5a-402f-a75e-9f701f45b8bf tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Request method failure captured: Jul 19 11:06:04.576722 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: request: GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 HTTP/1.1 Jul 19 11:06:04.576722 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept: application/json Jul 19 11:06:04.576722 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept-Encoding: identity Jul 19 11:06:04.576722 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Connection: close Jul 19 11:06:04.576722 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Length: 0 Jul 19 11:06:04.576722 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Type: application/json Jul 19 11:06:04.576722 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Host: 10.222.0.33 Jul 19 11:06:04.576722 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Openstack-System-Scope: None Jul 19 11:06:04.576722 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: User-Agent: python-urllib3/1.26.9 Jul 19 11:06:04.576722 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Auth-Token: gAAAAABi1o_BJ016weIAjEjud_yhYUf-1rKUqMCpyv3zNWY_c1DSJ6Pq7XYDHi4-5js1wFl6V91XYTMGb4k9zfAhVDb2Rvyf5qoTgPGamlrUFYVo5QJ1zSz2yULGNqkpiH8-08dpOp1RXriZerxkLjoGfFjO3t9Atloz6h5fT5SLMMSl-8sEWeM Jul 19 11:06:04.576722 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Id: None Jul 19 11:06:04.576722 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Name: None Jul 19 11:06:04.576722 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Identity-Status: Confirmed Jul 19 11:06:04.576722 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Is-Admin-Project: True Jul 19 11:06:04.576722 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Id: default Jul 19 11:06:04.576722 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Name: Default Jul 19 11:06:04.576722 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Id: 205683784b714ab399311aeca15d0cfa Jul 19 11:06:04.576722 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Name: tempest-DeleteServersTestJSON-1559915880 Jul 19 11:06:04.576722 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Role: member,reader Jul 19 11:06:04.576722 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Roles: member,reader Jul 19 11:06:04.577974 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/205683784b714ab399311aeca15d0cfa"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/205683784b714ab399311aeca15d0cfa"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/205683784b714ab399311aeca15d0cfa"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_205683784b714ab399311aeca15d0cfa"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:06:04.577974 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant: tempest-DeleteServersTestJSON-1559915880 Jul 19 11:06:04.577974 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Id: 205683784b714ab399311aeca15d0cfa Jul 19 11:06:04.577974 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Name: tempest-DeleteServersTestJSON-1559915880 Jul 19 11:06:04.577974 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User: tempest-DeleteServersTestJSON-1559915880-project Jul 19 11:06:04.577974 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Id: default Jul 19 11:06:04.577974 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Name: Default Jul 19 11:06:04.577974 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Id: f6cb55e4c41f4e6d94156c597a1baad8 Jul 19 11:06:04.577974 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Name: tempest-DeleteServersTestJSON-1559915880-project Jul 19 11:06:04.577974 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: method: > Jul 19 11:06:04.577974 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: exception: Instance 126c2860-81b1-42fc-8fbd-78178837f2c2 could not be found. Jul 19 11:06:04.577974 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: action_args: {'id': '126c2860-81b1-42fc-8fbd-78178837f2c2'} Jul 19 11:06:04.578686 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:06:04.578686 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:06:04.578686 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:06:04.578686 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:06:04.578686 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:06:04.578686 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:06:04.578686 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:06:04.578686 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:06:04.578686 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:06:04.578686 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:06:04.578686 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 126c2860-81b1-42fc-8fbd-78178837f2c2 could not be found. Jul 19 11:06:04.578686 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:06:04.578686 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:06:04.578686 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:06:04.578686 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:06:04.578686 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:06:04.578686 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:06:04.578686 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:06:04.578686 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:06:04.578686 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:06:04.579897 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:06:04.579897 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 19 11:06:04.579897 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 19 11:06:04.579897 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:06:04.579897 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:06:04.579897 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:06:04.579897 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:06:04.579897 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 126c2860-81b1-42fc-8fbd-78178837f2c2 could not be found. Jul 19 11:06:04.579897 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:06:04.579897 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:06:04.579897 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:06:04.579897 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:06:04.579897 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:06:04.579897 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:06:04.579897 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:06:04.579897 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:06:04.579897 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 126c2860-81b1-42fc-8fbd-78178837f2c2 could not be found. Jul 19 11:06:04.579897 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:06:04.581066 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-4fb0f2e6-ac5a-402f-a75e-9f701f45b8bf tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Returning 404 to user: Instance 126c2860-81b1-42fc-8fbd-78178837f2c2 could not be found. {{(pid=117849) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:06:04.582025 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-4fb0f2e6-ac5a-402f-a75e-9f701f45b8bf tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 404 len: 111 microversion: 2.1 time: 0.068526 Jul 19 11:06:04.585662 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 151/283] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:04 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 111 bytes in 72 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:06:04.721945 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-a69d18d9-db6a-4cdb-b4ce-23e2f561e668 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:04.723687 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-a69d18d9-db6a-4cdb-b4ce-23e2f561e668 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:04.732649 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a69d18d9-db6a-4cdb-b4ce-23e2f561e668 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:04.733150 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a69d18d9-db6a-4cdb-b4ce-23e2f561e668 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:04.733921 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a69d18d9-db6a-4cdb-b4ce-23e2f561e668 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:04.735019 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a69d18d9-db6a-4cdb-b4ce-23e2f561e668 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:04.751312 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.wsgi [None req-a69d18d9-db6a-4cdb-b4ce-23e2f561e668 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] HTTP exception thrown: Instance 126c2860-81b1-42fc-8fbd-78178837f2c2 could not be found. Jul 19 11:06:04.756465 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-a69d18d9-db6a-4cdb-b4ce-23e2f561e668 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Request method failure captured: Jul 19 11:06:04.756465 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: request: GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2/os-volume_attachments HTTP/1.1 Jul 19 11:06:04.756465 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept: application/json Jul 19 11:06:04.756465 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept-Encoding: identity Jul 19 11:06:04.756465 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Connection: close Jul 19 11:06:04.756465 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Length: 0 Jul 19 11:06:04.756465 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Type: application/json Jul 19 11:06:04.756465 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Host: 10.222.0.33 Jul 19 11:06:04.756465 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Openstack-System-Scope: None Jul 19 11:06:04.756465 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: User-Agent: python-urllib3/1.26.9 Jul 19 11:06:04.756465 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Auth-Token: gAAAAABi1o_BJ016weIAjEjud_yhYUf-1rKUqMCpyv3zNWY_c1DSJ6Pq7XYDHi4-5js1wFl6V91XYTMGb4k9zfAhVDb2Rvyf5qoTgPGamlrUFYVo5QJ1zSz2yULGNqkpiH8-08dpOp1RXriZerxkLjoGfFjO3t9Atloz6h5fT5SLMMSl-8sEWeM Jul 19 11:06:04.756465 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Id: None Jul 19 11:06:04.756465 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Name: None Jul 19 11:06:04.756465 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Identity-Status: Confirmed Jul 19 11:06:04.756465 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Is-Admin-Project: True Jul 19 11:06:04.756465 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Id: default Jul 19 11:06:04.756465 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Name: Default Jul 19 11:06:04.756465 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Id: 205683784b714ab399311aeca15d0cfa Jul 19 11:06:04.756465 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Name: tempest-DeleteServersTestJSON-1559915880 Jul 19 11:06:04.756465 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Role: member,reader Jul 19 11:06:04.756465 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Roles: member,reader Jul 19 11:06:04.758514 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/205683784b714ab399311aeca15d0cfa"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/205683784b714ab399311aeca15d0cfa"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/205683784b714ab399311aeca15d0cfa"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_205683784b714ab399311aeca15d0cfa"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:06:04.758514 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant: tempest-DeleteServersTestJSON-1559915880 Jul 19 11:06:04.758514 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Id: 205683784b714ab399311aeca15d0cfa Jul 19 11:06:04.758514 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Name: tempest-DeleteServersTestJSON-1559915880 Jul 19 11:06:04.758514 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User: tempest-DeleteServersTestJSON-1559915880-project Jul 19 11:06:04.758514 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Id: default Jul 19 11:06:04.758514 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Name: Default Jul 19 11:06:04.758514 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Id: f6cb55e4c41f4e6d94156c597a1baad8 Jul 19 11:06:04.758514 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Name: tempest-DeleteServersTestJSON-1559915880-project Jul 19 11:06:04.758514 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: method: > Jul 19 11:06:04.758514 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: exception: Instance 126c2860-81b1-42fc-8fbd-78178837f2c2 could not be found. Jul 19 11:06:04.758514 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: action_args: {'server_id': '126c2860-81b1-42fc-8fbd-78178837f2c2'} Jul 19 11:06:04.759151 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:06:04.759151 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:06:04.759151 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:06:04.759151 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:06:04.759151 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:06:04.759151 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:06:04.759151 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:06:04.759151 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:06:04.759151 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:06:04.759151 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:06:04.759151 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 126c2860-81b1-42fc-8fbd-78178837f2c2 could not be found. Jul 19 11:06:04.759151 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:06:04.759151 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:06:04.759151 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:06:04.759151 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:06:04.759151 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:06:04.759151 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:06:04.759151 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:06:04.759151 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:06:04.759151 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:06:04.760197 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:06:04.760197 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 19 11:06:04.760197 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 19 11:06:04.760197 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 19 11:06:04.760197 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 19 11:06:04.760197 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Jul 19 11:06:04.760197 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Jul 19 11:06:04.760197 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:06:04.760197 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:06:04.760197 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 126c2860-81b1-42fc-8fbd-78178837f2c2 could not be found. Jul 19 11:06:04.760197 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:06:04.760197 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:06:04.760197 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:06:04.760197 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:06:04.760197 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:06:04.760197 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:06:04.760197 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:06:04.760197 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:06:04.760197 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 126c2860-81b1-42fc-8fbd-78178837f2c2 could not be found. Jul 19 11:06:04.760197 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:06:04.763063 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-a69d18d9-db6a-4cdb-b4ce-23e2f561e668 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Returning 404 to user: Instance 126c2860-81b1-42fc-8fbd-78178837f2c2 could not be found. {{(pid=117848) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:06:04.764101 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-a69d18d9-db6a-4cdb-b4ce-23e2f561e668 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.048635 Jul 19 11:06:04.765325 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 133/284] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:06:04 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2/os-volume_attachments => generated 111 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:06:04.790339 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-b79c2f9c-c14d-42d2-a18d-56a9146660e4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:04.791176 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-b79c2f9c-c14d-42d2-a18d-56a9146660e4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:04.800737 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b79c2f9c-c14d-42d2-a18d-56a9146660e4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:04.802029 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b79c2f9c-c14d-42d2-a18d-56a9146660e4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:04.804278 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b79c2f9c-c14d-42d2-a18d-56a9146660e4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:04.805116 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b79c2f9c-c14d-42d2-a18d-56a9146660e4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:05.034959 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b79c2f9c-c14d-42d2-a18d-56a9146660e4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:05.043953 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-b79c2f9c-c14d-42d2-a18d-56a9146660e4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.257330 Jul 19 11:06:05.047901 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 152/285] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:04 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:05.187967 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-b7b8c709-c6cd-48c1-8e20-958640777a62 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:05.188883 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-b7b8c709-c6cd-48c1-8e20-958640777a62 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: b4e0099f-88fa-4929-94a2-ef54325bace7] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:05.197283 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b7b8c709-c6cd-48c1-8e20-958640777a62 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:05.199336 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b7b8c709-c6cd-48c1-8e20-958640777a62 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:05.200217 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b7b8c709-c6cd-48c1-8e20-958640777a62 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:05.201005 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b7b8c709-c6cd-48c1-8e20-958640777a62 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:05.222472 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.wsgi [None req-b7b8c709-c6cd-48c1-8e20-958640777a62 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] HTTP exception thrown: Instance b4e0099f-88fa-4929-94a2-ef54325bace7 could not be found. Jul 19 11:06:05.226446 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-b7b8c709-c6cd-48c1-8e20-958640777a62 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Request method failure captured: Jul 19 11:06:05.226446 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: request: GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 HTTP/1.1 Jul 19 11:06:05.226446 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept: application/json Jul 19 11:06:05.226446 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept-Encoding: identity Jul 19 11:06:05.226446 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Connection: close Jul 19 11:06:05.226446 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Length: 0 Jul 19 11:06:05.226446 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Type: application/json Jul 19 11:06:05.226446 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Host: 10.222.0.33 Jul 19 11:06:05.226446 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Openstack-System-Scope: None Jul 19 11:06:05.226446 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: User-Agent: python-urllib3/1.26.9 Jul 19 11:06:05.226446 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Auth-Token: gAAAAABi1o_ozfNFlPaQRnzwClF_YGouCVxQBGtd1GAF8cUkI_779udeLJz9DeLPRMUBCozNKBRJt0xWLObn2vT6qw8Ur5VvkyRmYxXd6ERfqPeNVZXH0ImmoGyDSzSRl0TEgsMLM70zEBhWBiIkNjGm1HOKzGmzWErhAUnsRBFnmcMnUlvMfI4 Jul 19 11:06:05.226446 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Id: None Jul 19 11:06:05.226446 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Name: None Jul 19 11:06:05.226446 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Identity-Status: Confirmed Jul 19 11:06:05.226446 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Is-Admin-Project: True Jul 19 11:06:05.226446 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Id: default Jul 19 11:06:05.226446 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Name: Default Jul 19 11:06:05.226446 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Id: 88dc732812fc4fdd8e9c307f3565ba58 Jul 19 11:06:05.226446 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Name: tempest-VolumesActionsTest-2018501907 Jul 19 11:06:05.226446 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Role: member,reader Jul 19 11:06:05.226446 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Roles: member,reader Jul 19 11:06:05.227670 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/88dc732812fc4fdd8e9c307f3565ba58"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/88dc732812fc4fdd8e9c307f3565ba58"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/88dc732812fc4fdd8e9c307f3565ba58"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_88dc732812fc4fdd8e9c307f3565ba58"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:06:05.227670 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant: tempest-VolumesActionsTest-2018501907 Jul 19 11:06:05.227670 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Id: 88dc732812fc4fdd8e9c307f3565ba58 Jul 19 11:06:05.227670 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Name: tempest-VolumesActionsTest-2018501907 Jul 19 11:06:05.227670 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User: tempest-VolumesActionsTest-2018501907-project Jul 19 11:06:05.227670 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Id: default Jul 19 11:06:05.227670 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Name: Default Jul 19 11:06:05.227670 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Id: e2293054dd2740e781825cbd068d74f7 Jul 19 11:06:05.227670 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Name: tempest-VolumesActionsTest-2018501907-project Jul 19 11:06:05.227670 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: method: > Jul 19 11:06:05.227670 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: exception: Instance b4e0099f-88fa-4929-94a2-ef54325bace7 could not be found. Jul 19 11:06:05.227670 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: action_args: {'id': 'b4e0099f-88fa-4929-94a2-ef54325bace7'} Jul 19 11:06:05.228397 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:06:05.228397 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:06:05.228397 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:06:05.228397 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:06:05.228397 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:06:05.228397 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:06:05.228397 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:06:05.228397 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:06:05.228397 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:06:05.228397 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:06:05.228397 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b4e0099f-88fa-4929-94a2-ef54325bace7 could not be found. Jul 19 11:06:05.228397 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:06:05.228397 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:06:05.228397 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:06:05.228397 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:06:05.228397 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:06:05.228397 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:06:05.228397 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:06:05.228397 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:06:05.228397 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:06:05.229544 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:06:05.229544 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 19 11:06:05.229544 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 19 11:06:05.229544 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:06:05.229544 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:06:05.229544 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:06:05.229544 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:06:05.229544 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b4e0099f-88fa-4929-94a2-ef54325bace7 could not be found. Jul 19 11:06:05.229544 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:06:05.229544 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:06:05.229544 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:06:05.229544 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:06:05.229544 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:06:05.229544 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:06:05.229544 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:06:05.229544 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:06:05.229544 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b4e0099f-88fa-4929-94a2-ef54325bace7 could not be found. Jul 19 11:06:05.229544 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:06:05.232144 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-b7b8c709-c6cd-48c1-8e20-958640777a62 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Returning 404 to user: Instance b4e0099f-88fa-4929-94a2-ef54325bace7 could not be found. {{(pid=117848) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:06:05.233671 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-b7b8c709-c6cd-48c1-8e20-958640777a62 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7" status: 404 len: 111 microversion: 2.1 time: 0.051114 Jul 19 11:06:05.234448 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 134/286] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:05 2022] GET /compute/v2.1/servers/b4e0099f-88fa-4929-94a2-ef54325bace7 => generated 111 bytes in 53 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:06:05.439622 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-10f1281c-4902-464d-b0b3-d41eb38439fa tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-932501370", "imageRef": "3085883e-2289-4314-aa47-9c41d4368b02", "flavorRef": "452", "networks": [{"uuid": "392cf9f9-e9a5-4cb6-9c0a-fce64784a4d2"}]}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:06:05.507061 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.network.neutron [None req-10f1281c-4902-464d-b0b3-d41eb38439fa tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] validate_networks() for [('392cf9f9-e9a5-4cb6-9c0a-fce64784a4d2', None, None, None, None, None)] {{(pid=117849) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 19 11:06:06.062005 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-8374ad02-3a43-4aa2-b232-5e0f6d748b65 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:06.062620 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-8374ad02-3a43-4aa2-b232-5e0f6d748b65 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:06.071790 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8374ad02-3a43-4aa2-b232-5e0f6d748b65 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:06.073133 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8374ad02-3a43-4aa2-b232-5e0f6d748b65 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:06.074343 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8374ad02-3a43-4aa2-b232-5e0f6d748b65 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:06.075571 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8374ad02-3a43-4aa2-b232-5e0f6d748b65 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:06.263789 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-8374ad02-3a43-4aa2-b232-5e0f6d748b65 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:06.271483 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-10f1281c-4902-464d-b0b3-d41eb38439fa tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Flavor limits 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 19 11:06:06.272037 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-10f1281c-4902-464d-b0b3-d41eb38439fa tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Image limits 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 19 11:06:06.272398 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-10f1281c-4902-464d-b0b3-d41eb38439fa tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Flavor pref 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 19 11:06:06.272854 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-10f1281c-4902-464d-b0b3-d41eb38439fa tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Image pref 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 19 11:06:06.273270 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-10f1281c-4902-464d-b0b3-d41eb38439fa tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 19 11:06:06.283670 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-8374ad02-3a43-4aa2-b232-5e0f6d748b65 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.226219 Jul 19 11:06:06.293636 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 135/287] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:06 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:06:06.298583 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-10f1281c-4902-464d-b0b3-d41eb38439fa tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:06.299065 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-10f1281c-4902-464d-b0b3-d41eb38439fa tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:06.300094 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-10f1281c-4902-464d-b0b3-d41eb38439fa tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:06.300655 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-10f1281c-4902-464d-b0b3-d41eb38439fa tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:06.302771 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-429b5a88-ff85-4a36-a678-f07ffe4d8f67 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:06.303324 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-429b5a88-ff85-4a36-a678-f07ffe4d8f67 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:06.320452 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-429b5a88-ff85-4a36-a678-f07ffe4d8f67 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:06.321087 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-429b5a88-ff85-4a36-a678-f07ffe4d8f67 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:06.321895 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-429b5a88-ff85-4a36-a678-f07ffe4d8f67 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:06.323851 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-429b5a88-ff85-4a36-a678-f07ffe4d8f67 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:06.341526 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.wsgi [None req-429b5a88-ff85-4a36-a678-f07ffe4d8f67 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] HTTP exception thrown: Instance 126c2860-81b1-42fc-8fbd-78178837f2c2 could not be found. Jul 19 11:06:06.343608 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-429b5a88-ff85-4a36-a678-f07ffe4d8f67 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Request method failure captured: Jul 19 11:06:06.343608 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: request: DELETE /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 HTTP/1.1 Jul 19 11:06:06.343608 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept: application/json Jul 19 11:06:06.343608 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept-Encoding: identity Jul 19 11:06:06.343608 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Connection: close Jul 19 11:06:06.343608 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Length: 0 Jul 19 11:06:06.343608 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Type: application/json Jul 19 11:06:06.343608 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Host: 10.222.0.33 Jul 19 11:06:06.343608 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Openstack-System-Scope: None Jul 19 11:06:06.343608 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: User-Agent: python-urllib3/1.26.9 Jul 19 11:06:06.343608 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Auth-Token: gAAAAABi1o_BJ016weIAjEjud_yhYUf-1rKUqMCpyv3zNWY_c1DSJ6Pq7XYDHi4-5js1wFl6V91XYTMGb4k9zfAhVDb2Rvyf5qoTgPGamlrUFYVo5QJ1zSz2yULGNqkpiH8-08dpOp1RXriZerxkLjoGfFjO3t9Atloz6h5fT5SLMMSl-8sEWeM Jul 19 11:06:06.343608 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Id: None Jul 19 11:06:06.343608 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Name: None Jul 19 11:06:06.343608 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Identity-Status: Confirmed Jul 19 11:06:06.343608 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Is-Admin-Project: True Jul 19 11:06:06.343608 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Id: default Jul 19 11:06:06.343608 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Name: Default Jul 19 11:06:06.343608 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Id: 205683784b714ab399311aeca15d0cfa Jul 19 11:06:06.343608 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Name: tempest-DeleteServersTestJSON-1559915880 Jul 19 11:06:06.343608 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Role: member,reader Jul 19 11:06:06.343608 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Roles: member,reader Jul 19 11:06:06.344870 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/205683784b714ab399311aeca15d0cfa"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/205683784b714ab399311aeca15d0cfa"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/205683784b714ab399311aeca15d0cfa"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_205683784b714ab399311aeca15d0cfa"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:06:06.344870 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant: tempest-DeleteServersTestJSON-1559915880 Jul 19 11:06:06.344870 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Id: 205683784b714ab399311aeca15d0cfa Jul 19 11:06:06.344870 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Name: tempest-DeleteServersTestJSON-1559915880 Jul 19 11:06:06.344870 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User: tempest-DeleteServersTestJSON-1559915880-project Jul 19 11:06:06.344870 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Id: default Jul 19 11:06:06.344870 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Name: Default Jul 19 11:06:06.344870 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Id: f6cb55e4c41f4e6d94156c597a1baad8 Jul 19 11:06:06.344870 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Name: tempest-DeleteServersTestJSON-1559915880-project Jul 19 11:06:06.344870 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: method: > Jul 19 11:06:06.344870 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: exception: Instance 126c2860-81b1-42fc-8fbd-78178837f2c2 could not be found. Jul 19 11:06:06.344870 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: action_args: {'id': '126c2860-81b1-42fc-8fbd-78178837f2c2'} Jul 19 11:06:06.345571 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:06:06.345571 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:06:06.345571 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:06:06.345571 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:06:06.345571 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:06:06.345571 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:06:06.345571 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:06:06.345571 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:06:06.345571 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:06:06.345571 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:06:06.345571 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 126c2860-81b1-42fc-8fbd-78178837f2c2 could not be found. Jul 19 11:06:06.345571 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:06:06.345571 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:06:06.345571 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:06:06.345571 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:06:06.345571 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:06:06.345571 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:06:06.345571 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:06:06.345571 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:06:06.345571 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:06:06.347443 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:06:06.347443 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 19 11:06:06.347443 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 19 11:06:06.347443 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 19 11:06:06.347443 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 19 11:06:06.347443 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:06:06.347443 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:06:06.347443 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:06:06.347443 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:06:06.347443 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 126c2860-81b1-42fc-8fbd-78178837f2c2 could not be found. Jul 19 11:06:06.347443 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:06:06.347443 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:06:06.347443 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:06:06.347443 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:06:06.347443 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:06:06.347443 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:06:06.347443 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:06:06.347443 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:06:06.347443 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 126c2860-81b1-42fc-8fbd-78178837f2c2 could not be found. Jul 19 11:06:06.347443 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:06:06.349839 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-429b5a88-ff85-4a36-a678-f07ffe4d8f67 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Returning 404 to user: Instance 126c2860-81b1-42fc-8fbd-78178837f2c2 could not be found. {{(pid=117848) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:06:06.349839 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-429b5a88-ff85-4a36-a678-f07ffe4d8f67 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "DELETE /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 404 len: 111 microversion: 2.1 time: 0.051972 Jul 19 11:06:06.349839 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 136/288] 10.222.0.33 () {58 vars in 1238 bytes} [Tue Jul 19 11:06:06 2022] DELETE /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 111 bytes in 56 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:06:06.351065 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-10f1281c-4902-464d-b0b3-d41eb38439fa tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Getting quotas for project 88dc732812fc4fdd8e9c307f3565ba58. Resources: {'cores', 'instances', 'ram'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:06:06.356233 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-d8960f72-2451-493e-bfaf-4e49f3c0fd75 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:06.358832 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-10f1281c-4902-464d-b0b3-d41eb38439fa tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Getting quotas for user e2293054dd2740e781825cbd068d74f7 and project 88dc732812fc4fdd8e9c307f3565ba58. Resources: {'cores', 'instances', 'ram'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:06:06.361729 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-d8960f72-2451-493e-bfaf-4e49f3c0fd75 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] [instance: 126c2860-81b1-42fc-8fbd-78178837f2c2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:06.372523 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d8960f72-2451-493e-bfaf-4e49f3c0fd75 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:06.373001 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d8960f72-2451-493e-bfaf-4e49f3c0fd75 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:06.373785 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d8960f72-2451-493e-bfaf-4e49f3c0fd75 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:06.374210 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d8960f72-2451-493e-bfaf-4e49f3c0fd75 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:06.385924 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-10f1281c-4902-464d-b0b3-d41eb38439fa tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Going to run 1 instances... {{(pid=117849) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 19 11:06:06.390020 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-10f1281c-4902-464d-b0b3-d41eb38439fa tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: 09c5a8d5-8225-497b-8e9b-21981fb151e2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='3085883e-2289-4314-aa47-9c41d4368b02',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117849) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 19 11:06:06.398404 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.wsgi [None req-d8960f72-2451-493e-bfaf-4e49f3c0fd75 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] HTTP exception thrown: Instance 126c2860-81b1-42fc-8fbd-78178837f2c2 could not be found. Jul 19 11:06:06.401934 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-d8960f72-2451-493e-bfaf-4e49f3c0fd75 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Request method failure captured: Jul 19 11:06:06.401934 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: request: GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 HTTP/1.1 Jul 19 11:06:06.401934 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept: application/json Jul 19 11:06:06.401934 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept-Encoding: identity Jul 19 11:06:06.401934 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Connection: close Jul 19 11:06:06.401934 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Length: 0 Jul 19 11:06:06.401934 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Type: application/json Jul 19 11:06:06.401934 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Host: 10.222.0.33 Jul 19 11:06:06.401934 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Openstack-System-Scope: None Jul 19 11:06:06.401934 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: User-Agent: python-urllib3/1.26.9 Jul 19 11:06:06.401934 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Auth-Token: gAAAAABi1o_BJ016weIAjEjud_yhYUf-1rKUqMCpyv3zNWY_c1DSJ6Pq7XYDHi4-5js1wFl6V91XYTMGb4k9zfAhVDb2Rvyf5qoTgPGamlrUFYVo5QJ1zSz2yULGNqkpiH8-08dpOp1RXriZerxkLjoGfFjO3t9Atloz6h5fT5SLMMSl-8sEWeM Jul 19 11:06:06.401934 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Id: None Jul 19 11:06:06.401934 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Name: None Jul 19 11:06:06.401934 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Identity-Status: Confirmed Jul 19 11:06:06.401934 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Is-Admin-Project: True Jul 19 11:06:06.401934 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Id: default Jul 19 11:06:06.401934 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Name: Default Jul 19 11:06:06.401934 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Id: 205683784b714ab399311aeca15d0cfa Jul 19 11:06:06.401934 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Name: tempest-DeleteServersTestJSON-1559915880 Jul 19 11:06:06.401934 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Role: member,reader Jul 19 11:06:06.401934 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Roles: member,reader Jul 19 11:06:06.403094 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/205683784b714ab399311aeca15d0cfa"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/205683784b714ab399311aeca15d0cfa"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/205683784b714ab399311aeca15d0cfa"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_205683784b714ab399311aeca15d0cfa"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:06:06.403094 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant: tempest-DeleteServersTestJSON-1559915880 Jul 19 11:06:06.403094 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Id: 205683784b714ab399311aeca15d0cfa Jul 19 11:06:06.403094 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Name: tempest-DeleteServersTestJSON-1559915880 Jul 19 11:06:06.403094 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User: tempest-DeleteServersTestJSON-1559915880-project Jul 19 11:06:06.403094 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Id: default Jul 19 11:06:06.403094 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Name: Default Jul 19 11:06:06.403094 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Id: f6cb55e4c41f4e6d94156c597a1baad8 Jul 19 11:06:06.403094 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Name: tempest-DeleteServersTestJSON-1559915880-project Jul 19 11:06:06.403094 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: method: > Jul 19 11:06:06.403094 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: exception: Instance 126c2860-81b1-42fc-8fbd-78178837f2c2 could not be found. Jul 19 11:06:06.403094 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: action_args: {'id': '126c2860-81b1-42fc-8fbd-78178837f2c2'} Jul 19 11:06:06.403724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:06:06.403724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:06:06.403724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:06:06.403724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:06:06.403724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:06:06.403724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:06:06.403724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:06:06.403724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:06:06.403724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:06:06.403724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:06:06.403724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 126c2860-81b1-42fc-8fbd-78178837f2c2 could not be found. Jul 19 11:06:06.403724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:06:06.403724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:06:06.403724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:06:06.403724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:06:06.403724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:06:06.403724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:06:06.403724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:06:06.403724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:06:06.403724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:06:06.404831 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:06:06.404831 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 19 11:06:06.404831 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 19 11:06:06.404831 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:06:06.404831 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:06:06.404831 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:06:06.404831 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:06:06.404831 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 126c2860-81b1-42fc-8fbd-78178837f2c2 could not be found. Jul 19 11:06:06.404831 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:06:06.404831 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:06:06.404831 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:06:06.404831 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:06:06.404831 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:06:06.404831 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:06:06.404831 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:06:06.404831 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:06:06.404831 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 126c2860-81b1-42fc-8fbd-78178837f2c2 could not be found. Jul 19 11:06:06.404831 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:06:06.408263 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-d8960f72-2451-493e-bfaf-4e49f3c0fd75 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] Returning 404 to user: Instance 126c2860-81b1-42fc-8fbd-78178837f2c2 could not be found. {{(pid=117848) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:06:06.411928 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-d8960f72-2451-493e-bfaf-4e49f3c0fd75 tempest-DeleteServersTestJSON-1559915880 tempest-DeleteServersTestJSON-1559915880-project] 10.222.0.33 "GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2" status: 404 len: 111 microversion: 2.1 time: 0.059349 Jul 19 11:06:06.412750 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 137/289] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:06 2022] GET /compute/v2.1/servers/126c2860-81b1-42fc-8fbd-78178837f2c2 => generated 111 bytes in 63 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:06:06.516941 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-10f1281c-4902-464d-b0b3-d41eb38439fa tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.081520 Jul 19 11:06:06.519039 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 153/290] 10.222.0.33 () {60 vars in 1146 bytes} [Tue Jul 19 11:06:05 2022] POST /compute/v2.1/servers => generated 380 bytes in 1084 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 19 11:06:06.527613 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-c7f45d60-8034-49c3-a1af-33e05c84c05c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:06.530904 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-c7f45d60-8034-49c3-a1af-33e05c84c05c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: 09c5a8d5-8225-497b-8e9b-21981fb151e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:06.703851 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-c7f45d60-8034-49c3-a1af-33e05c84c05c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:06.716882 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-c7f45d60-8034-49c3-a1af-33e05c84c05c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2" status: 200 len: 1213 microversion: 2.1 time: 0.189526 Jul 19 11:06:06.717279 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 138/291] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:06 2022] GET /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2 => generated 1213 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:07.301810 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-260cf5af-edf8-4690-b324-3adb6d79b401 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:07.302430 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-260cf5af-edf8-4690-b324-3adb6d79b401 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:07.313918 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-260cf5af-edf8-4690-b324-3adb6d79b401 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:07.314280 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-260cf5af-edf8-4690-b324-3adb6d79b401 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:07.320287 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-260cf5af-edf8-4690-b324-3adb6d79b401 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:07.321279 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-260cf5af-edf8-4690-b324-3adb6d79b401 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:07.744440 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-a5403665-0fd1-47e2-8231-847ad3c0e14f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:07.746249 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-a5403665-0fd1-47e2-8231-847ad3c0e14f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: 09c5a8d5-8225-497b-8e9b-21981fb151e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:07.760314 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a5403665-0fd1-47e2-8231-847ad3c0e14f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:07.760816 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a5403665-0fd1-47e2-8231-847ad3c0e14f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:07.761592 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a5403665-0fd1-47e2-8231-847ad3c0e14f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:07.762185 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a5403665-0fd1-47e2-8231-847ad3c0e14f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:07.835374 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-260cf5af-edf8-4690-b324-3adb6d79b401 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:07.846191 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-260cf5af-edf8-4690-b324-3adb6d79b401 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.546686 Jul 19 11:06:07.847052 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 154/292] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:07 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 550 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:07.901487 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-a5403665-0fd1-47e2-8231-847ad3c0e14f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:07.914989 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-a5403665-0fd1-47e2-8231-847ad3c0e14f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2" status: 200 len: 1265 microversion: 2.1 time: 0.185194 Jul 19 11:06:07.916162 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 139/293] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:07 2022] GET /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2 => generated 1265 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:08.863532 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-3eaa9629-982e-4058-8981-2cec84698cae tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:08.864186 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-3eaa9629-982e-4058-8981-2cec84698cae tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:08.871528 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3eaa9629-982e-4058-8981-2cec84698cae tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:08.886596 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3eaa9629-982e-4058-8981-2cec84698cae tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:08.887458 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3eaa9629-982e-4058-8981-2cec84698cae tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:08.887894 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3eaa9629-982e-4058-8981-2cec84698cae tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:08.933188 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-775dd142-d9da-4184-a1d4-e612b3e517bf tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:08.934432 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-775dd142-d9da-4184-a1d4-e612b3e517bf tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: 09c5a8d5-8225-497b-8e9b-21981fb151e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:08.954125 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-775dd142-d9da-4184-a1d4-e612b3e517bf tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:08.954571 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-775dd142-d9da-4184-a1d4-e612b3e517bf tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:08.959102 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-775dd142-d9da-4184-a1d4-e612b3e517bf tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:08.959735 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-775dd142-d9da-4184-a1d4-e612b3e517bf tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:09.254027 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-775dd142-d9da-4184-a1d4-e612b3e517bf tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:09.275809 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-775dd142-d9da-4184-a1d4-e612b3e517bf tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2" status: 200 len: 1271 microversion: 2.1 time: 0.344138 Jul 19 11:06:09.278276 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 140/294] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:08 2022] GET /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2 => generated 1271 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:06:09.340456 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-3eaa9629-982e-4058-8981-2cec84698cae tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:09.380884 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-3eaa9629-982e-4058-8981-2cec84698cae tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.521159 Jul 19 11:06:09.384762 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 155/295] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:08 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 526 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:10.295485 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-0a2dd1c2-4268-459f-9de9-c0a8076ab0f3 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:10.296904 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-0a2dd1c2-4268-459f-9de9-c0a8076ab0f3 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: 09c5a8d5-8225-497b-8e9b-21981fb151e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:10.307362 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0a2dd1c2-4268-459f-9de9-c0a8076ab0f3 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:10.308190 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0a2dd1c2-4268-459f-9de9-c0a8076ab0f3 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:10.312708 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0a2dd1c2-4268-459f-9de9-c0a8076ab0f3 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:10.314313 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0a2dd1c2-4268-459f-9de9-c0a8076ab0f3 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:10.407422 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-7fd78e40-ea51-4e19-80a9-f4254be6bba5 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:10.409291 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-7fd78e40-ea51-4e19-80a9-f4254be6bba5 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:10.421055 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7fd78e40-ea51-4e19-80a9-f4254be6bba5 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:10.421943 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7fd78e40-ea51-4e19-80a9-f4254be6bba5 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:10.423245 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7fd78e40-ea51-4e19-80a9-f4254be6bba5 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:10.423880 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7fd78e40-ea51-4e19-80a9-f4254be6bba5 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:10.625866 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-0a2dd1c2-4268-459f-9de9-c0a8076ab0f3 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:10.636772 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-0a2dd1c2-4268-459f-9de9-c0a8076ab0f3 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2" status: 200 len: 1271 microversion: 2.1 time: 0.345198 Jul 19 11:06:10.638442 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 141/296] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:10 2022] GET /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2 => generated 1271 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:10.845799 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-7fd78e40-ea51-4e19-80a9-f4254be6bba5 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:10.877294 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-7fd78e40-ea51-4e19-80a9-f4254be6bba5 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.476875 Jul 19 11:06:10.879314 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 156/297] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:10 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:06:11.656507 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-29be6738-88e5-4047-95a8-c6e5573fabf2 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:11.657273 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-29be6738-88e5-4047-95a8-c6e5573fabf2 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: 09c5a8d5-8225-497b-8e9b-21981fb151e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:11.670297 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-29be6738-88e5-4047-95a8-c6e5573fabf2 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:11.670738 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-29be6738-88e5-4047-95a8-c6e5573fabf2 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:11.672250 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-29be6738-88e5-4047-95a8-c6e5573fabf2 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:11.672876 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-29be6738-88e5-4047-95a8-c6e5573fabf2 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:11.895560 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-4e8022e5-e511-425a-a694-b68a535d76b9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:11.896890 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-4e8022e5-e511-425a-a694-b68a535d76b9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:11.909208 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4e8022e5-e511-425a-a694-b68a535d76b9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:11.909662 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4e8022e5-e511-425a-a694-b68a535d76b9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:11.910432 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4e8022e5-e511-425a-a694-b68a535d76b9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:11.910866 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4e8022e5-e511-425a-a694-b68a535d76b9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:12.212550 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-4e8022e5-e511-425a-a694-b68a535d76b9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:12.225154 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-4e8022e5-e511-425a-a694-b68a535d76b9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.333449 Jul 19 11:06:12.226137 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 157/298] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:11 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:12.305791 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-29be6738-88e5-4047-95a8-c6e5573fabf2 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:12.314590 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-29be6738-88e5-4047-95a8-c6e5573fabf2 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2" status: 200 len: 1313 microversion: 2.1 time: 0.662390 Jul 19 11:06:12.315164 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 142/299] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:11 2022] GET /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2 => generated 1313 bytes in 664 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:12.507756 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-54829686-bbac-4a12-aa75-3f7a83604af4 req-4a6617e6-8655-4d02-8f52-f1e7be9d116e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "09c5a8d5-8225-497b-8e9b-21981fb151e2", "tag": "c37fa729-0e82-499d-b4ea-1c6da8bfd3a4"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:06:12.533385 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-54829686-bbac-4a12-aa75-3f7a83604af4 req-4a6617e6-8655-4d02-8f52-f1e7be9d116e service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:12.534006 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-54829686-bbac-4a12-aa75-3f7a83604af4 req-4a6617e6-8655-4d02-8f52-f1e7be9d116e service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:12.547981 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-54829686-bbac-4a12-aa75-3f7a83604af4 req-4a6617e6-8655-4d02-8f52-f1e7be9d116e service nova] Creating event network-changed:c37fa729-0e82-499d-b4ea-1c6da8bfd3a4 for instance 09c5a8d5-8225-497b-8e9b-21981fb151e2 on hv-cinder-maste Jul 19 11:06:12.553429 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-54829686-bbac-4a12-aa75-3f7a83604af4 req-4a6617e6-8655-4d02-8f52-f1e7be9d116e service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.058860 Jul 19 11:06:12.554689 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 143/300] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:06:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:06:12.578297 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-1be50364-1dd0-4fd9-930a-42f0988f9be2 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Action: 'create', calling method: .version_select at 0x7f04aa4baee0>, body: {"keypair": {"name": "tempest-TestShelveInstance-261123105", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLkr7niDdQawxSETVSKsUBdXELSIASb64JI4WC/TeA3yIgVI/BFHuPJzrzjaIAMEVN06NZUvQavRfn0YQiDFVYzJEDPEKF5ed+67id5LDg7gRABjXbV6+3JsXMOY/bGeZA=="}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:06:12.593506 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-1be50364-1dd0-4fd9-930a-42f0988f9be2 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Getting quotas for project 375de853371a422f883980296a46ad43. Resources: {'key_pairs'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:06:12.601875 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-1be50364-1dd0-4fd9-930a-42f0988f9be2 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Getting quotas for user 3ece5061796245a1b2a6bef1d9d6323a and project 375de853371a422f883980296a46ad43. Resources: {'key_pairs'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:06:12.626812 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-1be50364-1dd0-4fd9-930a-42f0988f9be2 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "POST /compute/v2.1/os-keypairs" status: 200 len: 396 microversion: 2.1 time: 0.333490 Jul 19 11:06:12.627426 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 158/301] 10.222.0.33 () {60 vars in 1158 bytes} [Tue Jul 19 11:06:12 2022] POST /compute/v2.1/os-keypairs => generated 396 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:06:13.246846 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-cfad4114-d5d8-435e-a451-5a2e5e951fb8 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:13.247657 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-cfad4114-d5d8-435e-a451-5a2e5e951fb8 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:13.275423 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cfad4114-d5d8-435e-a451-5a2e5e951fb8 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:13.276042 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cfad4114-d5d8-435e-a451-5a2e5e951fb8 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:13.277267 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cfad4114-d5d8-435e-a451-5a2e5e951fb8 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:13.277912 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cfad4114-d5d8-435e-a451-5a2e5e951fb8 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:13.333739 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-c78b65e2-db11-4d22-a277-f44c6be61d6c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:13.334231 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-c78b65e2-db11-4d22-a277-f44c6be61d6c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: 09c5a8d5-8225-497b-8e9b-21981fb151e2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:13.345635 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c78b65e2-db11-4d22-a277-f44c6be61d6c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:13.349300 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c78b65e2-db11-4d22-a277-f44c6be61d6c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:13.350142 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c78b65e2-db11-4d22-a277-f44c6be61d6c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:13.351136 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c78b65e2-db11-4d22-a277-f44c6be61d6c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:13.664532 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-cfad4114-d5d8-435e-a451-5a2e5e951fb8 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:13.683700 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-c78b65e2-db11-4d22-a277-f44c6be61d6c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:13.693252 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-c78b65e2-db11-4d22-a277-f44c6be61d6c tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2" status: 200 len: 1313 microversion: 2.1 time: 0.364756 Jul 19 11:06:13.693953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-cfad4114-d5d8-435e-a451-5a2e5e951fb8 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.434337 Jul 19 11:06:13.694863 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 144/302] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:13 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:13.704836 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 159/303] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:13 2022] GET /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2 => generated 1313 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:06:14.725160 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-0f66d14b-aeee-4928-84f8-ee3f85c50465 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:14.726696 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-978d6928-ce2c-46d1-85df-6a74514f5b80 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:14.727044 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-978d6928-ce2c-46d1-85df-6a74514f5b80 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:14.727962 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-0f66d14b-aeee-4928-84f8-ee3f85c50465 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: 09c5a8d5-8225-497b-8e9b-21981fb151e2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:14.742987 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-978d6928-ce2c-46d1-85df-6a74514f5b80 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:14.744186 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-978d6928-ce2c-46d1-85df-6a74514f5b80 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:14.745040 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0f66d14b-aeee-4928-84f8-ee3f85c50465 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:14.745951 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0f66d14b-aeee-4928-84f8-ee3f85c50465 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:14.746695 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-978d6928-ce2c-46d1-85df-6a74514f5b80 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:14.747970 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0f66d14b-aeee-4928-84f8-ee3f85c50465 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:14.748780 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-978d6928-ce2c-46d1-85df-6a74514f5b80 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:14.750781 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0f66d14b-aeee-4928-84f8-ee3f85c50465 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:15.073729 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-0f66d14b-aeee-4928-84f8-ee3f85c50465 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:15.085612 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-0f66d14b-aeee-4928-84f8-ee3f85c50465 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2" status: 200 len: 1313 microversion: 2.1 time: 0.372662 Jul 19 11:06:15.086203 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 160/304] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:14 2022] GET /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2 => generated 1313 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:15.460428 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-978d6928-ce2c-46d1-85df-6a74514f5b80 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:15.464199 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=100 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:06:15.468934 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-978d6928-ce2c-46d1-85df-6a74514f5b80 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.759988 Jul 19 11:06:15.484207 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 145/305] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:14 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 774 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:16.102913 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-1423ad71-b2eb-4250-8c39-16e6069d0582 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:16.103521 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-1423ad71-b2eb-4250-8c39-16e6069d0582 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: 09c5a8d5-8225-497b-8e9b-21981fb151e2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:16.112953 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1423ad71-b2eb-4250-8c39-16e6069d0582 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:16.114769 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1423ad71-b2eb-4250-8c39-16e6069d0582 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:16.116017 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1423ad71-b2eb-4250-8c39-16e6069d0582 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:16.116456 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1423ad71-b2eb-4250-8c39-16e6069d0582 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:16.317482 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell0:SELECT=2 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:06:16.328173 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-1423ad71-b2eb-4250-8c39-16e6069d0582 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:16.339623 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-1423ad71-b2eb-4250-8c39-16e6069d0582 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2" status: 200 len: 1313 microversion: 2.1 time: 0.240829 Jul 19 11:06:16.349879 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 161/306] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:16 2022] GET /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2 => generated 1313 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:16.492387 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-3712074e-77ca-4f99-963a-c35899444ebb tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:16.493080 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-3712074e-77ca-4f99-963a-c35899444ebb tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:16.512945 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3712074e-77ca-4f99-963a-c35899444ebb tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:16.513553 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3712074e-77ca-4f99-963a-c35899444ebb tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:16.514510 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3712074e-77ca-4f99-963a-c35899444ebb tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:16.515274 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3712074e-77ca-4f99-963a-c35899444ebb tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:16.709448 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell0:SELECT=2 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:06:16.713887 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-3712074e-77ca-4f99-963a-c35899444ebb tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:16.725981 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-3712074e-77ca-4f99-963a-c35899444ebb tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.243454 Jul 19 11:06:16.727782 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 146/307] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:16 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:17.357396 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-33afea72-20ad-49a3-8ee8-d8824f3499c4 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:17.361225 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-33afea72-20ad-49a3-8ee8-d8824f3499c4 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: 09c5a8d5-8225-497b-8e9b-21981fb151e2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:17.368057 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-33afea72-20ad-49a3-8ee8-d8824f3499c4 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:17.368618 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-33afea72-20ad-49a3-8ee8-d8824f3499c4 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:17.370112 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-33afea72-20ad-49a3-8ee8-d8824f3499c4 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:17.370548 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-33afea72-20ad-49a3-8ee8-d8824f3499c4 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:17.531027 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-33afea72-20ad-49a3-8ee8-d8824f3499c4 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:17.542725 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-33afea72-20ad-49a3-8ee8-d8824f3499c4 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2" status: 200 len: 1313 microversion: 2.1 time: 0.189730 Jul 19 11:06:17.543843 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 162/308] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:17 2022] GET /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2 => generated 1313 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:17.744529 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-f3bdba3d-31ce-4aac-8a0a-28d0fb44a521 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:17.745898 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-f3bdba3d-31ce-4aac-8a0a-28d0fb44a521 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:17.753212 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f3bdba3d-31ce-4aac-8a0a-28d0fb44a521 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:17.753913 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f3bdba3d-31ce-4aac-8a0a-28d0fb44a521 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:17.754671 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f3bdba3d-31ce-4aac-8a0a-28d0fb44a521 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:17.755256 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f3bdba3d-31ce-4aac-8a0a-28d0fb44a521 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:18.267711 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-f3bdba3d-31ce-4aac-8a0a-28d0fb44a521 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:18.277917 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-f3bdba3d-31ce-4aac-8a0a-28d0fb44a521 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.537229 Jul 19 11:06:18.287621 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 147/309] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:17 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 548 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:18.534456 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-05961ce0-5202-4053-89f3-4fe0eac92b98 req-617b8320-91ac-40ad-96ac-42449b6b0aa0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "09c5a8d5-8225-497b-8e9b-21981fb151e2", "name": "network-vif-plugged", "status": "completed", "tag": "c37fa729-0e82-499d-b4ea-1c6da8bfd3a4"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:06:18.546514 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-05961ce0-5202-4053-89f3-4fe0eac92b98 req-617b8320-91ac-40ad-96ac-42449b6b0aa0 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:18.547251 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-05961ce0-5202-4053-89f3-4fe0eac92b98 req-617b8320-91ac-40ad-96ac-42449b6b0aa0 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:18.560452 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-05961ce0-5202-4053-89f3-4fe0eac92b98 req-617b8320-91ac-40ad-96ac-42449b6b0aa0 service nova] Creating event network-vif-plugged:c37fa729-0e82-499d-b4ea-1c6da8bfd3a4 for instance 09c5a8d5-8225-497b-8e9b-21981fb151e2 on hv-cinder-maste Jul 19 11:06:18.562050 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-5b01df32-08b1-42b1-bdb2-5f9b1e17cfac tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:18.562738 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-5b01df32-08b1-42b1-bdb2-5f9b1e17cfac tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: 09c5a8d5-8225-497b-8e9b-21981fb151e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:18.567009 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-05961ce0-5202-4053-89f3-4fe0eac92b98 req-617b8320-91ac-40ad-96ac-42449b6b0aa0 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037968 Jul 19 11:06:18.568352 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 163/310] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:06:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:06:18.571273 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5b01df32-08b1-42b1-bdb2-5f9b1e17cfac tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:18.571736 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5b01df32-08b1-42b1-bdb2-5f9b1e17cfac tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:18.572523 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5b01df32-08b1-42b1-bdb2-5f9b1e17cfac tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:18.572991 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5b01df32-08b1-42b1-bdb2-5f9b1e17cfac tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:19.199422 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-5b01df32-08b1-42b1-bdb2-5f9b1e17cfac tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:19.210220 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-5b01df32-08b1-42b1-bdb2-5f9b1e17cfac tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2" status: 200 len: 1313 microversion: 2.1 time: 0.652224 Jul 19 11:06:19.210972 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 148/311] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:18 2022] GET /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2 => generated 1313 bytes in 654 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:19.297225 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-86a7fd5f-3feb-4f88-b5c5-d3efd8a9e462 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:19.297971 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-86a7fd5f-3feb-4f88-b5c5-d3efd8a9e462 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:19.304491 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-86a7fd5f-3feb-4f88-b5c5-d3efd8a9e462 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:19.305171 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-86a7fd5f-3feb-4f88-b5c5-d3efd8a9e462 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:19.306040 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-86a7fd5f-3feb-4f88-b5c5-d3efd8a9e462 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:19.306484 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-86a7fd5f-3feb-4f88-b5c5-d3efd8a9e462 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:19.461998 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-86a7fd5f-3feb-4f88-b5c5-d3efd8a9e462 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:19.469563 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-86a7fd5f-3feb-4f88-b5c5-d3efd8a9e462 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.178047 Jul 19 11:06:19.470135 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 164/312] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:19 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:20.229416 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-1f75a285-56d0-425a-ae8f-7d176e56ed8f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:20.231333 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-1f75a285-56d0-425a-ae8f-7d176e56ed8f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: 09c5a8d5-8225-497b-8e9b-21981fb151e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:20.238388 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1f75a285-56d0-425a-ae8f-7d176e56ed8f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:20.238990 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1f75a285-56d0-425a-ae8f-7d176e56ed8f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:20.239868 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1f75a285-56d0-425a-ae8f-7d176e56ed8f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:20.242723 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1f75a285-56d0-425a-ae8f-7d176e56ed8f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:20.487653 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-153e63f4-7509-4b9f-883d-9bd0af640520 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:20.489168 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-153e63f4-7509-4b9f-883d-9bd0af640520 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:20.496651 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-153e63f4-7509-4b9f-883d-9bd0af640520 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:20.497651 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-153e63f4-7509-4b9f-883d-9bd0af640520 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:20.498670 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-153e63f4-7509-4b9f-883d-9bd0af640520 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:20.499323 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-153e63f4-7509-4b9f-883d-9bd0af640520 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:20.532448 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=100 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:06:20.535979 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-1f75a285-56d0-425a-ae8f-7d176e56ed8f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:20.555806 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-1f75a285-56d0-425a-ae8f-7d176e56ed8f tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2" status: 200 len: 1313 microversion: 2.1 time: 0.331348 Jul 19 11:06:20.570429 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 149/313] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:20 2022] GET /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2 => generated 1313 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:20.682285 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-153e63f4-7509-4b9f-883d-9bd0af640520 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:20.690271 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-153e63f4-7509-4b9f-883d-9bd0af640520 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.207777 Jul 19 11:06:20.699532 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 165/314] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:20 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:06:21.573062 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-89cdbaee-4c3b-4595-81b0-5e6f7eb6eac3 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:21.574016 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-89cdbaee-4c3b-4595-81b0-5e6f7eb6eac3 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: 09c5a8d5-8225-497b-8e9b-21981fb151e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:21.583213 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-89cdbaee-4c3b-4595-81b0-5e6f7eb6eac3 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:21.584157 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-89cdbaee-4c3b-4595-81b0-5e6f7eb6eac3 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:21.585219 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-89cdbaee-4c3b-4595-81b0-5e6f7eb6eac3 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:21.586032 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-89cdbaee-4c3b-4595-81b0-5e6f7eb6eac3 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:21.707335 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-30b2ddfd-83f8-481e-acac-9e67409c4902 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:21.708521 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-30b2ddfd-83f8-481e-acac-9e67409c4902 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:21.715605 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-30b2ddfd-83f8-481e-acac-9e67409c4902 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:21.716274 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-30b2ddfd-83f8-481e-acac-9e67409c4902 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:21.717085 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-30b2ddfd-83f8-481e-acac-9e67409c4902 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:21.717526 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-30b2ddfd-83f8-481e-acac-9e67409c4902 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:21.805090 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-89cdbaee-4c3b-4595-81b0-5e6f7eb6eac3 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:21.815073 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-89cdbaee-4c3b-4595-81b0-5e6f7eb6eac3 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2" status: 200 len: 1498 microversion: 2.1 time: 0.247039 Jul 19 11:06:21.817942 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 150/315] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:21 2022] GET /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2 => generated 1498 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:21.921672 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-30b2ddfd-83f8-481e-acac-9e67409c4902 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:21.926967 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-30b2ddfd-83f8-481e-acac-9e67409c4902 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.224344 Jul 19 11:06:21.933055 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 166/316] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:21 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:22.445286 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-faad3f21-8e70-4e18-b9f6-969cec987302 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Action: 'create', calling method: .version_select at 0x7f04aa726550>, body: {"keypair": {"name": "tempest-keypair-1751795436", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLPTyOB0BQgVpnuVrOSMQfMD657auLOLt0rzWk5kCIl6x0913rU17cVj40Ktx5IcdO+TtyLjaeIYeCt0p67wpcXHV08PEF/mV4aDMX1XhG2bYDJg7Y7Ze7HjHqAWa8ezFg=="}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:06:22.460932 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.quota [None req-faad3f21-8e70-4e18-b9f6-969cec987302 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Getting quotas for project 54b5133f102c4606ac7981542949b178. Resources: {'key_pairs'} {{(pid=117848) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:06:22.465188 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.quota [None req-faad3f21-8e70-4e18-b9f6-969cec987302 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Getting quotas for user c0b6f4cfede44b2aa8f4a8dfc1e211d6 and project 54b5133f102c4606ac7981542949b178. Resources: {'key_pairs'} {{(pid=117848) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:06:22.488484 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-faad3f21-8e70-4e18-b9f6-969cec987302 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.170159 Jul 19 11:06:22.489128 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 151/317] 10.222.0.33 () {60 vars in 1158 bytes} [Tue Jul 19 11:06:22 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:06:22.588553 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-084c2d9a-470f-4c3a-be8c-0fb567424d5b tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:22.589200 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-084c2d9a-470f-4c3a-be8c-0fb567424d5b tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: 09c5a8d5-8225-497b-8e9b-21981fb151e2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:22.596031 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-084c2d9a-470f-4c3a-be8c-0fb567424d5b tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:22.596613 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-084c2d9a-470f-4c3a-be8c-0fb567424d5b tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:22.597580 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-084c2d9a-470f-4c3a-be8c-0fb567424d5b tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:22.598738 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-084c2d9a-470f-4c3a-be8c-0fb567424d5b tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:22.627615 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell0:SELECT=10 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:06:22.650453 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-084c2d9a-470f-4c3a-be8c-0fb567424d5b tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: 09c5a8d5-8225-497b-8e9b-21981fb151e2] Going to try to terminate instance {{(pid=117849) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 19 11:06:22.749132 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_api:SELECT=94,nova_api:UPDATE=2,nova_api:INSERT=4 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:06:22.760067 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-084c2d9a-470f-4c3a-be8c-0fb567424d5b tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "DELETE /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2" status: 204 len: 0 microversion: 2.1 time: 0.174896 Jul 19 11:06:22.761089 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 167/318] 10.222.0.33 () {58 vars in 1238 bytes} [Tue Jul 19 11:06:22 2022] DELETE /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 19 11:06:22.772213 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-f1e214fe-0bb5-4b78-8c7a-97defd4d1ade tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:22.773485 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-f1e214fe-0bb5-4b78-8c7a-97defd4d1ade tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: 09c5a8d5-8225-497b-8e9b-21981fb151e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:22.781913 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f1e214fe-0bb5-4b78-8c7a-97defd4d1ade tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:22.782467 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f1e214fe-0bb5-4b78-8c7a-97defd4d1ade tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:22.783327 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f1e214fe-0bb5-4b78-8c7a-97defd4d1ade tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:22.783778 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f1e214fe-0bb5-4b78-8c7a-97defd4d1ade tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:22.954092 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-394884d8-0c3a-4fd6-850a-2693adf7e1b2 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:22.954834 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-394884d8-0c3a-4fd6-850a-2693adf7e1b2 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:22.967977 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-394884d8-0c3a-4fd6-850a-2693adf7e1b2 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:22.968139 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-394884d8-0c3a-4fd6-850a-2693adf7e1b2 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:22.968948 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-394884d8-0c3a-4fd6-850a-2693adf7e1b2 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:22.969380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-394884d8-0c3a-4fd6-850a-2693adf7e1b2 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:23.163907 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-394884d8-0c3a-4fd6-850a-2693adf7e1b2 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:23.178148 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-394884d8-0c3a-4fd6-850a-2693adf7e1b2 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.232296 Jul 19 11:06:23.180377 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 168/319] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:22 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:06:23.226108 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-f1e214fe-0bb5-4b78-8c7a-97defd4d1ade tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:23.236862 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-f1e214fe-0bb5-4b78-8c7a-97defd4d1ade tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2" status: 200 len: 1504 microversion: 2.1 time: 0.470321 Jul 19 11:06:23.237967 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 152/320] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:22 2022] GET /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2 => generated 1504 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:24.191015 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-ca63ec5f-2d79-4884-a087-48e0cad71846 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-694256311", "imageRef": "3085883e-2289-4314-aa47-9c41d4368b02", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "f82d9bcd-5d24-448d-9b1c-e69a9c19f6e7"}], "security_groups": [{"name": "tempest-securitygroup--1444024443"}], "key_name": "tempest-keypair-1751795436", "user_data": "}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:06:24.199705 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-37fee460-b222-41aa-bbb2-c0ee1f3e44ee tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:24.200307 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-37fee460-b222-41aa-bbb2-c0ee1f3e44ee tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:24.214489 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-37fee460-b222-41aa-bbb2-c0ee1f3e44ee tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:24.215236 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-37fee460-b222-41aa-bbb2-c0ee1f3e44ee tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:24.216007 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-37fee460-b222-41aa-bbb2-c0ee1f3e44ee tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:24.216438 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-37fee460-b222-41aa-bbb2-c0ee1f3e44ee tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:24.348088 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-37fee460-b222-41aa-bbb2-c0ee1f3e44ee tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:24.357646 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-37fee460-b222-41aa-bbb2-c0ee1f3e44ee tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.161601 Jul 19 11:06:24.358522 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 153/321] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:24 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:06:24.363935 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-2c2c1cbc-ee18-4a16-b939-461f509533d7 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:24.364659 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-2c2c1cbc-ee18-4a16-b939-461f509533d7 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: 09c5a8d5-8225-497b-8e9b-21981fb151e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:24.371643 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2c2c1cbc-ee18-4a16-b939-461f509533d7 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:24.372165 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2c2c1cbc-ee18-4a16-b939-461f509533d7 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:24.373457 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2c2c1cbc-ee18-4a16-b939-461f509533d7 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:24.373965 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2c2c1cbc-ee18-4a16-b939-461f509533d7 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:24.506620 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-2c2c1cbc-ee18-4a16-b939-461f509533d7 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:24.514787 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-2c2c1cbc-ee18-4a16-b939-461f509533d7 tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2" status: 200 len: 1504 microversion: 2.1 time: 0.154971 Jul 19 11:06:24.515367 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 154/322] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:24 2022] GET /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2 => generated 1504 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:25.375579 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-46a18e5c-29a2-48fa-8e0a-c7a7d0c001e3 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:25.376411 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-46a18e5c-29a2-48fa-8e0a-c7a7d0c001e3 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:25.386478 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-46a18e5c-29a2-48fa-8e0a-c7a7d0c001e3 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:25.387031 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-46a18e5c-29a2-48fa-8e0a-c7a7d0c001e3 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:25.387913 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-46a18e5c-29a2-48fa-8e0a-c7a7d0c001e3 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:25.388449 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-46a18e5c-29a2-48fa-8e0a-c7a7d0c001e3 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:25.586313 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-46a18e5c-29a2-48fa-8e0a-c7a7d0c001e3 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:25.606365 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-46a18e5c-29a2-48fa-8e0a-c7a7d0c001e3 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.234751 Jul 19 11:06:25.607905 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 155/323] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:25 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:25.613770 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-9efa3668-aaf5-4965-8376-3e6f89230b67 req-2a711e87-3ccf-43f9-a751-aa41d38f45c8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "09c5a8d5-8225-497b-8e9b-21981fb151e2", "name": "network-vif-deleted", "tag": "c37fa729-0e82-499d-b4ea-1c6da8bfd3a4"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:06:25.628347 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-9efa3668-aaf5-4965-8376-3e6f89230b67 req-2a711e87-3ccf-43f9-a751-aa41d38f45c8 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:25.629249 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-9efa3668-aaf5-4965-8376-3e6f89230b67 req-2a711e87-3ccf-43f9-a751-aa41d38f45c8 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:25.645435 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-9efa3668-aaf5-4965-8376-3e6f89230b67 req-2a711e87-3ccf-43f9-a751-aa41d38f45c8 service nova] Creating event network-vif-deleted:c37fa729-0e82-499d-b4ea-1c6da8bfd3a4 for instance 09c5a8d5-8225-497b-8e9b-21981fb151e2 on hv-cinder-maste Jul 19 11:06:25.650756 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-9efa3668-aaf5-4965-8376-3e6f89230b67 req-2a711e87-3ccf-43f9-a751-aa41d38f45c8 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041605 Jul 19 11:06:25.651636 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 156/324] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:06:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:06:25.658321 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-96417256-d8f9-423d-b20d-60916ea1d12e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:25.659054 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-96417256-d8f9-423d-b20d-60916ea1d12e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: 09c5a8d5-8225-497b-8e9b-21981fb151e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:25.667108 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-96417256-d8f9-423d-b20d-60916ea1d12e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:25.667556 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-96417256-d8f9-423d-b20d-60916ea1d12e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:25.668322 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-96417256-d8f9-423d-b20d-60916ea1d12e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:25.668963 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-96417256-d8f9-423d-b20d-60916ea1d12e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:25.704249 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-96417256-d8f9-423d-b20d-60916ea1d12e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:25.704776 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-96417256-d8f9-423d-b20d-60916ea1d12e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:25.705164 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.objects.instance [None req-96417256-d8f9-423d-b20d-60916ea1d12e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lazy-loading 'fault' on Instance uuid 09c5a8d5-8225-497b-8e9b-21981fb151e2 {{(pid=117848) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 19 11:06:25.751405 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-96417256-d8f9-423d-b20d-60916ea1d12e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e2293054dd2740e781825cbd068d74f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88dc732812fc4fdd8e9c307f3565ba58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:25.759861 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-96417256-d8f9-423d-b20d-60916ea1d12e tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2" status: 200 len: 1303 microversion: 2.1 time: 0.106808 Jul 19 11:06:25.760446 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 157/325] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:25 2022] GET /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2 => generated 1303 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:26.437230 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.network.neutron [None req-ca63ec5f-2d79-4884-a087-48e0cad71846 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] validate_networks() for [('f82d9bcd-5d24-448d-9b1c-e69a9c19f6e7', None, None, None, None, None)] {{(pid=117849) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 19 11:06:26.627256 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-3e77ea73-949f-47ed-b7fa-1925149fd98b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:26.628724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-3e77ea73-949f-47ed-b7fa-1925149fd98b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:26.631321 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_api:SELECT=87,nova_api:INSERT=1 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:06:26.643004 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3e77ea73-949f-47ed-b7fa-1925149fd98b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:26.643598 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3e77ea73-949f-47ed-b7fa-1925149fd98b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:26.644964 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3e77ea73-949f-47ed-b7fa-1925149fd98b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:26.646118 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3e77ea73-949f-47ed-b7fa-1925149fd98b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:26.823032 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-3e77ea73-949f-47ed-b7fa-1925149fd98b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:26.833436 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-3e77ea73-949f-47ed-b7fa-1925149fd98b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.211986 Jul 19 11:06:26.834189 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 158/326] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:26 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:26.849581 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-37d811bd-d925-45a5-bf38-f455303f637d tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:26.850742 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-37d811bd-d925-45a5-bf38-f455303f637d tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] [instance: 09c5a8d5-8225-497b-8e9b-21981fb151e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:26.857360 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-37d811bd-d925-45a5-bf38-f455303f637d tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:26.857925 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-37d811bd-d925-45a5-bf38-f455303f637d tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:26.858906 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-37d811bd-d925-45a5-bf38-f455303f637d tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:26.859500 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-37d811bd-d925-45a5-bf38-f455303f637d tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:26.868686 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-ca63ec5f-2d79-4884-a087-48e0cad71846 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Flavor limits 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 19 11:06:26.869369 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-ca63ec5f-2d79-4884-a087-48e0cad71846 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Image limits 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 19 11:06:26.870052 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-ca63ec5f-2d79-4884-a087-48e0cad71846 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Flavor pref 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 19 11:06:26.870917 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-ca63ec5f-2d79-4884-a087-48e0cad71846 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Image pref 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 19 11:06:26.871399 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-ca63ec5f-2d79-4884-a087-48e0cad71846 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 19 11:06:26.874891 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.wsgi [None req-37d811bd-d925-45a5-bf38-f455303f637d tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] HTTP exception thrown: Instance 09c5a8d5-8225-497b-8e9b-21981fb151e2 could not be found. Jul 19 11:06:26.876542 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-37d811bd-d925-45a5-bf38-f455303f637d tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Request method failure captured: Jul 19 11:06:26.876542 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: request: GET /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2 HTTP/1.1 Jul 19 11:06:26.876542 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept: application/json Jul 19 11:06:26.876542 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept-Encoding: identity Jul 19 11:06:26.876542 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Connection: close Jul 19 11:06:26.876542 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Length: 0 Jul 19 11:06:26.876542 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Type: application/json Jul 19 11:06:26.876542 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Host: 10.222.0.33 Jul 19 11:06:26.876542 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Openstack-System-Scope: None Jul 19 11:06:26.876542 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: User-Agent: python-urllib3/1.26.9 Jul 19 11:06:26.876542 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Auth-Token: gAAAAABi1o_ozfNFlPaQRnzwClF_YGouCVxQBGtd1GAF8cUkI_779udeLJz9DeLPRMUBCozNKBRJt0xWLObn2vT6qw8Ur5VvkyRmYxXd6ERfqPeNVZXH0ImmoGyDSzSRl0TEgsMLM70zEBhWBiIkNjGm1HOKzGmzWErhAUnsRBFnmcMnUlvMfI4 Jul 19 11:06:26.876542 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Id: None Jul 19 11:06:26.876542 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Name: None Jul 19 11:06:26.876542 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Identity-Status: Confirmed Jul 19 11:06:26.876542 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Is-Admin-Project: True Jul 19 11:06:26.876542 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Id: default Jul 19 11:06:26.876542 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Name: Default Jul 19 11:06:26.876542 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Id: 88dc732812fc4fdd8e9c307f3565ba58 Jul 19 11:06:26.876542 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Name: tempest-VolumesActionsTest-2018501907 Jul 19 11:06:26.876542 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Role: member,reader Jul 19 11:06:26.876542 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Roles: member,reader Jul 19 11:06:26.878026 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/88dc732812fc4fdd8e9c307f3565ba58"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/88dc732812fc4fdd8e9c307f3565ba58"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/88dc732812fc4fdd8e9c307f3565ba58"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_88dc732812fc4fdd8e9c307f3565ba58"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:06:26.878026 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant: tempest-VolumesActionsTest-2018501907 Jul 19 11:06:26.878026 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Id: 88dc732812fc4fdd8e9c307f3565ba58 Jul 19 11:06:26.878026 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Name: tempest-VolumesActionsTest-2018501907 Jul 19 11:06:26.878026 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User: tempest-VolumesActionsTest-2018501907-project Jul 19 11:06:26.878026 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Id: default Jul 19 11:06:26.878026 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Name: Default Jul 19 11:06:26.878026 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Id: e2293054dd2740e781825cbd068d74f7 Jul 19 11:06:26.878026 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Name: tempest-VolumesActionsTest-2018501907-project Jul 19 11:06:26.878026 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: method: > Jul 19 11:06:26.878026 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: exception: Instance 09c5a8d5-8225-497b-8e9b-21981fb151e2 could not be found. Jul 19 11:06:26.878026 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: action_args: {'id': '09c5a8d5-8225-497b-8e9b-21981fb151e2'} Jul 19 11:06:26.878758 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:06:26.878758 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:06:26.878758 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:06:26.878758 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:06:26.878758 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:06:26.878758 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:06:26.878758 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:06:26.878758 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:06:26.878758 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:06:26.878758 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:06:26.878758 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 09c5a8d5-8225-497b-8e9b-21981fb151e2 could not be found. Jul 19 11:06:26.878758 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:06:26.878758 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:06:26.878758 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:06:26.878758 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:06:26.878758 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:06:26.878758 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:06:26.878758 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:06:26.878758 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:06:26.878758 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:06:26.879860 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:06:26.879860 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 19 11:06:26.879860 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 19 11:06:26.879860 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:06:26.879860 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:06:26.879860 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:06:26.879860 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:06:26.879860 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 09c5a8d5-8225-497b-8e9b-21981fb151e2 could not be found. Jul 19 11:06:26.879860 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:06:26.879860 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:06:26.879860 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:06:26.879860 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:06:26.879860 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:06:26.879860 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:06:26.879860 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:06:26.879860 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:06:26.879860 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 09c5a8d5-8225-497b-8e9b-21981fb151e2 could not be found. Jul 19 11:06:26.879860 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:06:26.881405 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-37d811bd-d925-45a5-bf38-f455303f637d tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] Returning 404 to user: Instance 09c5a8d5-8225-497b-8e9b-21981fb151e2 could not be found. {{(pid=117848) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:06:26.881405 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-37d811bd-d925-45a5-bf38-f455303f637d tempest-VolumesActionsTest-2018501907 tempest-VolumesActionsTest-2018501907-project] 10.222.0.33 "GET /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2" status: 404 len: 111 microversion: 2.1 time: 0.032742 Jul 19 11:06:26.881405 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 159/327] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:26 2022] GET /compute/v2.1/servers/09c5a8d5-8225-497b-8e9b-21981fb151e2 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:06:26.887649 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ca63ec5f-2d79-4884-a087-48e0cad71846 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:26.888207 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ca63ec5f-2d79-4884-a087-48e0cad71846 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:26.889185 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ca63ec5f-2d79-4884-a087-48e0cad71846 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:26.889838 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ca63ec5f-2d79-4884-a087-48e0cad71846 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:26.908381 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-ca63ec5f-2d79-4884-a087-48e0cad71846 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Getting quotas for project 54b5133f102c4606ac7981542949b178. Resources: {'cores', 'instances', 'ram'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:06:26.913231 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-ca63ec5f-2d79-4884-a087-48e0cad71846 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Getting quotas for user c0b6f4cfede44b2aa8f4a8dfc1e211d6 and project 54b5133f102c4606ac7981542949b178. Resources: {'cores', 'instances', 'ram'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:06:26.924381 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-ca63ec5f-2d79-4884-a087-48e0cad71846 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Going to run 1 instances... {{(pid=117849) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 19 11:06:26.926721 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-ca63ec5f-2d79-4884-a087-48e0cad71846 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='3085883e-2289-4314-aa47-9c41d4368b02',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117849) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 19 11:06:26.973144 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-ca63ec5f-2d79-4884-a087-48e0cad71846 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 2.786235 Jul 19 11:06:26.974014 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 169/328] 10.222.0.33 () {60 vars in 1146 bytes} [Tue Jul 19 11:06:24 2022] POST /compute/v2.1/servers => generated 402 bytes in 2788 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 19 11:06:26.984566 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-8d2e37cf-1e91-4c9e-8d72-ac6344407567 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:26.985231 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-8d2e37cf-1e91-4c9e-8d72-ac6344407567 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:27.046664 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-8d2e37cf-1e91-4c9e-8d72-ac6344407567 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:27.056099 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-8d2e37cf-1e91-4c9e-8d72-ac6344407567 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1237 microversion: 2.1 time: 0.075100 Jul 19 11:06:27.058141 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 160/329] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:26 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1237 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:27.850771 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-46092617-5680-4188-b6de-cff4f6471a5c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:27.851376 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-46092617-5680-4188-b6de-cff4f6471a5c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:27.857736 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-46092617-5680-4188-b6de-cff4f6471a5c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:27.858184 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-46092617-5680-4188-b6de-cff4f6471a5c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:27.858959 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-46092617-5680-4188-b6de-cff4f6471a5c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:27.859391 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-46092617-5680-4188-b6de-cff4f6471a5c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:28.010319 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-46092617-5680-4188-b6de-cff4f6471a5c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:28.020966 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-46092617-5680-4188-b6de-cff4f6471a5c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.174115 Jul 19 11:06:28.022063 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 170/330] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:27 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:28.078391 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-355f15f8-6f5b-42b0-bbd8-7fa71418a7ba tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:28.080078 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-355f15f8-6f5b-42b0-bbd8-7fa71418a7ba tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:28.088015 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-355f15f8-6f5b-42b0-bbd8-7fa71418a7ba tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:28.088469 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-355f15f8-6f5b-42b0-bbd8-7fa71418a7ba tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:28.089411 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-355f15f8-6f5b-42b0-bbd8-7fa71418a7ba tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:28.090192 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-355f15f8-6f5b-42b0-bbd8-7fa71418a7ba tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:28.163401 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-355f15f8-6f5b-42b0-bbd8-7fa71418a7ba tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:28.173164 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-355f15f8-6f5b-42b0-bbd8-7fa71418a7ba tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1233 microversion: 2.1 time: 0.099547 Jul 19 11:06:28.173746 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 161/331] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:28 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1233 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:29.039385 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-f9d41806-03dc-4e44-8ea8-13a22eceb2bf tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:29.039987 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-f9d41806-03dc-4e44-8ea8-13a22eceb2bf tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:29.046017 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f9d41806-03dc-4e44-8ea8-13a22eceb2bf tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:29.046463 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f9d41806-03dc-4e44-8ea8-13a22eceb2bf tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:29.047222 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f9d41806-03dc-4e44-8ea8-13a22eceb2bf tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:29.047658 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f9d41806-03dc-4e44-8ea8-13a22eceb2bf tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:29.176662 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-f9d41806-03dc-4e44-8ea8-13a22eceb2bf tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:29.185627 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-f9d41806-03dc-4e44-8ea8-13a22eceb2bf tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1521 microversion: 2.1 time: 0.150528 Jul 19 11:06:29.186574 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 171/332] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:29 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1521 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:06:29.190902 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-6f7477b0-944c-4881-9d95-9f5888086e47 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:29.191548 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-6f7477b0-944c-4881-9d95-9f5888086e47 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:29.198557 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6f7477b0-944c-4881-9d95-9f5888086e47 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:29.199018 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6f7477b0-944c-4881-9d95-9f5888086e47 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:29.199854 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6f7477b0-944c-4881-9d95-9f5888086e47 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:29.200297 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6f7477b0-944c-4881-9d95-9f5888086e47 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:29.264211 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-6f7477b0-944c-4881-9d95-9f5888086e47 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:29.271306 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-6f7477b0-944c-4881-9d95-9f5888086e47 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1289 microversion: 2.1 time: 0.084152 Jul 19 11:06:29.272566 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 162/333] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:29 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1289 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:06:30.203379 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-a3a36772-7830-44ec-89a2-34f65478e7f6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:30.203905 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-a3a36772-7830-44ec-89a2-34f65478e7f6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:30.210428 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a3a36772-7830-44ec-89a2-34f65478e7f6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:30.210919 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a3a36772-7830-44ec-89a2-34f65478e7f6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:30.211665 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a3a36772-7830-44ec-89a2-34f65478e7f6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:30.212095 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a3a36772-7830-44ec-89a2-34f65478e7f6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:30.289201 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-4cf821f4-f2cd-47d3-9e2f-391aeb2e76b6 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:30.290381 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-4cf821f4-f2cd-47d3-9e2f-391aeb2e76b6 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:30.298696 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4cf821f4-f2cd-47d3-9e2f-391aeb2e76b6 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:30.299182 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4cf821f4-f2cd-47d3-9e2f-391aeb2e76b6 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:30.300023 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4cf821f4-f2cd-47d3-9e2f-391aeb2e76b6 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:30.300619 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4cf821f4-f2cd-47d3-9e2f-391aeb2e76b6 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:30.355924 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-a3a36772-7830-44ec-89a2-34f65478e7f6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:30.368730 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-4cf821f4-f2cd-47d3-9e2f-391aeb2e76b6 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:30.369348 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-a3a36772-7830-44ec-89a2-34f65478e7f6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1501 microversion: 2.1 time: 0.169759 Jul 19 11:06:30.370344 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 172/334] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:30 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1501 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:30.378377 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-4cf821f4-f2cd-47d3-9e2f-391aeb2e76b6 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1297 microversion: 2.1 time: 0.093698 Jul 19 11:06:30.378949 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 163/335] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:30 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1297 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:06:30.388094 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-05913f2c-4972-4a34-9025-97484dca3634 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:30.389039 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-05913f2c-4972-4a34-9025-97484dca3634 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:30.397604 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-05913f2c-4972-4a34-9025-97484dca3634 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:30.398308 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-05913f2c-4972-4a34-9025-97484dca3634 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:30.399301 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-05913f2c-4972-4a34-9025-97484dca3634 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:30.399891 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-05913f2c-4972-4a34-9025-97484dca3634 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:30.621578 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-05913f2c-4972-4a34-9025-97484dca3634 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:30.630579 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-05913f2c-4972-4a34-9025-97484dca3634 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1517 microversion: 2.1 time: 0.246268 Jul 19 11:06:30.631998 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 173/336] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:30 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1517 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:31.409962 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-93b395f6-c597-4703-9f49-e88b57dd1529 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:31.410575 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-93b395f6-c597-4703-9f49-e88b57dd1529 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:31.417697 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-93b395f6-c597-4703-9f49-e88b57dd1529 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:31.419327 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-93b395f6-c597-4703-9f49-e88b57dd1529 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:31.420266 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-93b395f6-c597-4703-9f49-e88b57dd1529 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:31.420881 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-93b395f6-c597-4703-9f49-e88b57dd1529 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:31.583756 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-93b395f6-c597-4703-9f49-e88b57dd1529 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:31.591750 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-93b395f6-c597-4703-9f49-e88b57dd1529 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1363 microversion: 2.1 time: 0.200351 Jul 19 11:06:31.592325 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 164/337] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:31 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1363 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:31.913692 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-522a617a-ad70-4d57-87fc-e56b6318ba1d req-60c7e1cc-3668-4899-bc07-a83b6db83fc4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "940e8a5c-b365-40c2-8361-fbb677969487", "tag": "d928944f-b596-4faa-b70f-efdde2d1f07e"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:06:31.930107 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-522a617a-ad70-4d57-87fc-e56b6318ba1d req-60c7e1cc-3668-4899-bc07-a83b6db83fc4 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:31.930794 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-522a617a-ad70-4d57-87fc-e56b6318ba1d req-60c7e1cc-3668-4899-bc07-a83b6db83fc4 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:31.954221 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-522a617a-ad70-4d57-87fc-e56b6318ba1d req-60c7e1cc-3668-4899-bc07-a83b6db83fc4 service nova] Creating event network-changed:d928944f-b596-4faa-b70f-efdde2d1f07e for instance 940e8a5c-b365-40c2-8361-fbb677969487 on hv-cinder-maste Jul 19 11:06:31.961275 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-522a617a-ad70-4d57-87fc-e56b6318ba1d req-60c7e1cc-3668-4899-bc07-a83b6db83fc4 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.051309 Jul 19 11:06:31.962023 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 174/338] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:06:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:06:32.078263 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-b5bce918-9872-4b46-92ba-b5c223839c6e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:06:32.079599 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-b5bce918-9872-4b46-92ba-b5c223839c6e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:32.087376 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b5bce918-9872-4b46-92ba-b5c223839c6e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:32.088121 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b5bce918-9872-4b46-92ba-b5c223839c6e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:32.089003 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b5bce918-9872-4b46-92ba-b5c223839c6e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:32.089932 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b5bce918-9872-4b46-92ba-b5c223839c6e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:32.110793 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=100 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:06:32.118010 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.objects.instance [None req-b5bce918-9872-4b46-92ba-b5c223839c6e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lazy-loading 'flavor' on Instance uuid c3ea5a95-376a-4482-b664-f069e803e258 {{(pid=117848) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 19 11:06:32.238801 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-b5bce918-9872-4b46-92ba-b5c223839c6e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "POST /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258/action" status: 200 len: 32 microversion: 2.1 time: 0.165287 Jul 19 11:06:32.239700 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 165/339] 10.222.0.33 () {60 vars in 1277 bytes} [Tue Jul 19 11:06:32 2022] POST /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258/action => generated 32 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 19 11:06:32.249721 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-44d5fd8a-b1c0-4e86-a7a6-9a2a7b5db0df tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:32.250423 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-44d5fd8a-b1c0-4e86-a7a6-9a2a7b5db0df tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:32.257135 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-44d5fd8a-b1c0-4e86-a7a6-9a2a7b5db0df tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:32.257844 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-44d5fd8a-b1c0-4e86-a7a6-9a2a7b5db0df tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:32.258821 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-44d5fd8a-b1c0-4e86-a7a6-9a2a7b5db0df tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:32.259492 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-44d5fd8a-b1c0-4e86-a7a6-9a2a7b5db0df tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:32.617208 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-0ef4b029-e691-45bf-b0e9-e0a5d938d047 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:32.617840 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-0ef4b029-e691-45bf-b0e9-e0a5d938d047 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:32.627698 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0ef4b029-e691-45bf-b0e9-e0a5d938d047 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:32.628526 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0ef4b029-e691-45bf-b0e9-e0a5d938d047 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:32.629541 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0ef4b029-e691-45bf-b0e9-e0a5d938d047 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:32.629983 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0ef4b029-e691-45bf-b0e9-e0a5d938d047 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:32.635559 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-44d5fd8a-b1c0-4e86-a7a6-9a2a7b5db0df tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:32.644536 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-44d5fd8a-b1c0-4e86-a7a6-9a2a7b5db0df tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1523 microversion: 2.1 time: 0.399266 Jul 19 11:06:32.645773 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 175/340] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:32 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1523 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:06:32.854225 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-0ef4b029-e691-45bf-b0e9-e0a5d938d047 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:32.862271 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-0ef4b029-e691-45bf-b0e9-e0a5d938d047 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1363 microversion: 2.1 time: 0.255981 Jul 19 11:06:32.862845 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 166/341] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:32 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1363 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:33.663513 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-f8dbd762-d4ac-457b-9ddd-f376fd34d43c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:33.664222 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-f8dbd762-d4ac-457b-9ddd-f376fd34d43c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:33.670599 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f8dbd762-d4ac-457b-9ddd-f376fd34d43c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:33.671048 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f8dbd762-d4ac-457b-9ddd-f376fd34d43c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:33.671827 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f8dbd762-d4ac-457b-9ddd-f376fd34d43c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:33.672265 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f8dbd762-d4ac-457b-9ddd-f376fd34d43c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:33.813313 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-f8dbd762-d4ac-457b-9ddd-f376fd34d43c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:33.821587 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-f8dbd762-d4ac-457b-9ddd-f376fd34d43c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1523 microversion: 2.1 time: 0.162421 Jul 19 11:06:33.822159 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 176/342] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:33 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1523 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:33.878998 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-dff24747-b18f-4450-9166-10326153f130 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:33.879612 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-dff24747-b18f-4450-9166-10326153f130 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:33.886153 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dff24747-b18f-4450-9166-10326153f130 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:33.886637 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dff24747-b18f-4450-9166-10326153f130 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:33.887397 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dff24747-b18f-4450-9166-10326153f130 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:33.887834 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dff24747-b18f-4450-9166-10326153f130 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:34.017835 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-dff24747-b18f-4450-9166-10326153f130 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:34.025722 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-dff24747-b18f-4450-9166-10326153f130 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1363 microversion: 2.1 time: 0.150553 Jul 19 11:06:34.026288 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 167/343] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:33 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1363 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:34.837613 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-6cae42e9-9659-46ae-bbd2-1920170b0d73 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:34.838266 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-6cae42e9-9659-46ae-bbd2-1920170b0d73 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:34.844225 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6cae42e9-9659-46ae-bbd2-1920170b0d73 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:34.844601 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6cae42e9-9659-46ae-bbd2-1920170b0d73 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:34.845390 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6cae42e9-9659-46ae-bbd2-1920170b0d73 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:34.845826 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6cae42e9-9659-46ae-bbd2-1920170b0d73 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:34.973809 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-6cae42e9-9659-46ae-bbd2-1920170b0d73 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:34.981883 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-6cae42e9-9659-46ae-bbd2-1920170b0d73 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1523 microversion: 2.1 time: 0.148090 Jul 19 11:06:34.982441 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 177/344] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:34 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1523 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:06:35.042056 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-f0d33c2c-ea8d-4632-a2bb-9dca797f4882 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:35.042656 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-f0d33c2c-ea8d-4632-a2bb-9dca797f4882 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:35.049065 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f0d33c2c-ea8d-4632-a2bb-9dca797f4882 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:35.049510 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f0d33c2c-ea8d-4632-a2bb-9dca797f4882 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:35.050259 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f0d33c2c-ea8d-4632-a2bb-9dca797f4882 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:35.050688 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f0d33c2c-ea8d-4632-a2bb-9dca797f4882 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:35.176094 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-f0d33c2c-ea8d-4632-a2bb-9dca797f4882 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:35.184008 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-f0d33c2c-ea8d-4632-a2bb-9dca797f4882 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1363 microversion: 2.1 time: 0.146268 Jul 19 11:06:35.184564 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 168/345] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:35 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1363 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:35.999423 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-ac8bb3da-2a9c-44c5-8af4-9b62644c8499 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:36.000024 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-ac8bb3da-2a9c-44c5-8af4-9b62644c8499 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:36.006004 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ac8bb3da-2a9c-44c5-8af4-9b62644c8499 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:36.006487 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ac8bb3da-2a9c-44c5-8af4-9b62644c8499 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:36.007242 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ac8bb3da-2a9c-44c5-8af4-9b62644c8499 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:36.007681 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ac8bb3da-2a9c-44c5-8af4-9b62644c8499 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:36.134901 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-ac8bb3da-2a9c-44c5-8af4-9b62644c8499 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:36.142242 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-ac8bb3da-2a9c-44c5-8af4-9b62644c8499 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1523 microversion: 2.1 time: 0.146897 Jul 19 11:06:36.142803 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 178/346] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:35 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1523 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:36.199776 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-0b58e9c1-dcc7-4d2c-b38e-5f0d5d222816 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:36.200408 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-0b58e9c1-dcc7-4d2c-b38e-5f0d5d222816 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:36.206857 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0b58e9c1-dcc7-4d2c-b38e-5f0d5d222816 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:36.207302 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0b58e9c1-dcc7-4d2c-b38e-5f0d5d222816 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:36.208043 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0b58e9c1-dcc7-4d2c-b38e-5f0d5d222816 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:36.208472 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0b58e9c1-dcc7-4d2c-b38e-5f0d5d222816 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:36.339637 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-0b58e9c1-dcc7-4d2c-b38e-5f0d5d222816 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:36.346983 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-0b58e9c1-dcc7-4d2c-b38e-5f0d5d222816 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1363 microversion: 2.1 time: 0.150974 Jul 19 11:06:36.347547 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 169/347] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:36 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1363 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:37.159343 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-e9f7dc79-7658-4b60-80d1-658811a81a40 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:37.159949 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-e9f7dc79-7658-4b60-80d1-658811a81a40 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:37.161978 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell0:SELECT=2 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:06:37.162699 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell0:SELECT=6 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:06:37.168356 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e9f7dc79-7658-4b60-80d1-658811a81a40 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:37.168846 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e9f7dc79-7658-4b60-80d1-658811a81a40 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:37.169606 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e9f7dc79-7658-4b60-80d1-658811a81a40 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:37.170068 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e9f7dc79-7658-4b60-80d1-658811a81a40 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:37.185001 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-cbb81314-117f-44ca-927b-9c8f01364fea tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:37.185488 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-cbb81314-117f-44ca-927b-9c8f01364fea tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:37.187528 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell0:SELECT=6 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:06:37.192832 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cbb81314-117f-44ca-927b-9c8f01364fea tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:37.193279 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cbb81314-117f-44ca-927b-9c8f01364fea tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:37.194538 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cbb81314-117f-44ca-927b-9c8f01364fea tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:37.194967 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cbb81314-117f-44ca-927b-9c8f01364fea tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:37.218548 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-cbb81314-117f-44ca-927b-9c8f01364fea tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326/ips" status: 200 len: 152 microversion: 2.20 time: 0.037335 Jul 19 11:06:37.219196 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 170/348] 10.222.0.33 () {60 vars in 1288 bytes} [Tue Jul 19 11:06:37 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326/ips => generated 152 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 19 11:06:37.298478 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-e9f7dc79-7658-4b60-80d1-658811a81a40 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:37.305951 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-e9f7dc79-7658-4b60-80d1-658811a81a40 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1523 microversion: 2.1 time: 0.150421 Jul 19 11:06:37.306518 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 179/349] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:37 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1523 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:37.364564 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-4275cc4d-6491-4a68-91da-ef2eb5610fe1 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:37.365263 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-4275cc4d-6491-4a68-91da-ef2eb5610fe1 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:37.371806 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4275cc4d-6491-4a68-91da-ef2eb5610fe1 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:37.372996 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4275cc4d-6491-4a68-91da-ef2eb5610fe1 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:37.373872 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4275cc4d-6491-4a68-91da-ef2eb5610fe1 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:37.374787 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4275cc4d-6491-4a68-91da-ef2eb5610fe1 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:37.535397 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-4275cc4d-6491-4a68-91da-ef2eb5610fe1 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:37.543136 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-4275cc4d-6491-4a68-91da-ef2eb5610fe1 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1363 microversion: 2.1 time: 0.183254 Jul 19 11:06:37.543692 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 171/350] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:37 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1363 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:38.322581 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-916504bc-8640-4493-87ae-5472a49e8f56 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:38.323178 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-916504bc-8640-4493-87ae-5472a49e8f56 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:38.329113 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-916504bc-8640-4493-87ae-5472a49e8f56 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:38.329560 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-916504bc-8640-4493-87ae-5472a49e8f56 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:38.330307 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-916504bc-8640-4493-87ae-5472a49e8f56 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:38.330738 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-916504bc-8640-4493-87ae-5472a49e8f56 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:38.458412 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-916504bc-8640-4493-87ae-5472a49e8f56 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:38.466446 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-916504bc-8640-4493-87ae-5472a49e8f56 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1523 microversion: 2.1 time: 0.147741 Jul 19 11:06:38.467015 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 180/351] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:38 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1523 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:38.559258 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-b8cd0dee-c484-45bd-8360-ec6c241878bf tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:38.559897 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-b8cd0dee-c484-45bd-8360-ec6c241878bf tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:38.566462 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b8cd0dee-c484-45bd-8360-ec6c241878bf tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:38.566916 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b8cd0dee-c484-45bd-8360-ec6c241878bf tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:38.567688 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b8cd0dee-c484-45bd-8360-ec6c241878bf tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:38.568131 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b8cd0dee-c484-45bd-8360-ec6c241878bf tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:38.698305 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-b8cd0dee-c484-45bd-8360-ec6c241878bf tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:38.705856 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-b8cd0dee-c484-45bd-8360-ec6c241878bf tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1363 microversion: 2.1 time: 0.150484 Jul 19 11:06:38.706423 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 172/352] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:38 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1363 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:39.483758 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-e377960b-19b6-4f92-8afd-f77a62da04bc tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:39.484390 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-e377960b-19b6-4f92-8afd-f77a62da04bc tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:39.490437 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e377960b-19b6-4f92-8afd-f77a62da04bc tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:39.490890 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e377960b-19b6-4f92-8afd-f77a62da04bc tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:39.491647 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e377960b-19b6-4f92-8afd-f77a62da04bc tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:39.492119 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e377960b-19b6-4f92-8afd-f77a62da04bc tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:39.507793 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:06:39.626667 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-e377960b-19b6-4f92-8afd-f77a62da04bc tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:39.634327 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-e377960b-19b6-4f92-8afd-f77a62da04bc tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1523 microversion: 2.1 time: 0.154441 Jul 19 11:06:39.634936 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 181/353] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:39 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1523 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:39.722944 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-f84fb721-0ac5-42bf-9ddf-1b02c72e60fb tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:39.723566 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-f84fb721-0ac5-42bf-9ddf-1b02c72e60fb tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:39.731436 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f84fb721-0ac5-42bf-9ddf-1b02c72e60fb tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:39.731877 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f84fb721-0ac5-42bf-9ddf-1b02c72e60fb tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:39.732634 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f84fb721-0ac5-42bf-9ddf-1b02c72e60fb tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:39.733251 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f84fb721-0ac5-42bf-9ddf-1b02c72e60fb tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:39.862640 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-f84fb721-0ac5-42bf-9ddf-1b02c72e60fb tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:39.870208 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-f84fb721-0ac5-42bf-9ddf-1b02c72e60fb tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1363 microversion: 2.1 time: 0.151221 Jul 19 11:06:39.870741 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 173/354] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:39 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1363 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:40.500655 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-2f62f16e-1bae-40fc-90b8-e08dcf840689 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:06:40.501818 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-2f62f16e-1bae-40fc-90b8-e08dcf840689 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:40.507955 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2f62f16e-1bae-40fc-90b8-e08dcf840689 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:40.508400 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2f62f16e-1bae-40fc-90b8-e08dcf840689 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:40.510041 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2f62f16e-1bae-40fc-90b8-e08dcf840689 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:40.510478 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2f62f16e-1bae-40fc-90b8-e08dcf840689 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:40.537055 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.objects.instance [None req-2f62f16e-1bae-40fc-90b8-e08dcf840689 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lazy-loading 'flavor' on Instance uuid f52fbfae-577b-403d-a4d3-9a4b79ae2326 {{(pid=117849) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 19 11:06:40.651615 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-ce28f7bf-2c43-4204-9145-f2eb503c8be7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:40.652280 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-ce28f7bf-2c43-4204-9145-f2eb503c8be7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:40.658963 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ce28f7bf-2c43-4204-9145-f2eb503c8be7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:40.659415 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ce28f7bf-2c43-4204-9145-f2eb503c8be7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:40.660283 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ce28f7bf-2c43-4204-9145-f2eb503c8be7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:40.660761 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ce28f7bf-2c43-4204-9145-f2eb503c8be7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:40.835916 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-ce28f7bf-2c43-4204-9145-f2eb503c8be7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:40.845803 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-ce28f7bf-2c43-4204-9145-f2eb503c8be7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1523 microversion: 2.1 time: 0.198372 Jul 19 11:06:40.856558 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 174/355] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:40 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1523 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:06:40.890110 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-562e9cd2-1c4b-4205-8c4c-955f89e07e1e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:40.891311 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-562e9cd2-1c4b-4205-8c4c-955f89e07e1e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:40.900210 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-562e9cd2-1c4b-4205-8c4c-955f89e07e1e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:40.900941 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-562e9cd2-1c4b-4205-8c4c-955f89e07e1e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:40.901968 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-562e9cd2-1c4b-4205-8c4c-955f89e07e1e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:40.902553 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-562e9cd2-1c4b-4205-8c4c-955f89e07e1e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:41.064485 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-562e9cd2-1c4b-4205-8c4c-955f89e07e1e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:41.077457 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-562e9cd2-1c4b-4205-8c4c-955f89e07e1e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1363 microversion: 2.1 time: 0.193637 Jul 19 11:06:41.078421 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 175/356] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:40 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1363 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:06:41.863515 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-07d25505-4079-4573-80e0-a0d184a01fb0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:41.864275 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-07d25505-4079-4573-80e0-a0d184a01fb0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:41.871156 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-07d25505-4079-4573-80e0-a0d184a01fb0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:41.871600 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-07d25505-4079-4573-80e0-a0d184a01fb0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:41.872347 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-07d25505-4079-4573-80e0-a0d184a01fb0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:41.873064 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-07d25505-4079-4573-80e0-a0d184a01fb0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:42.027129 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-07d25505-4079-4573-80e0-a0d184a01fb0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:42.034627 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-07d25505-4079-4573-80e0-a0d184a01fb0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1523 microversion: 2.1 time: 0.175074 Jul 19 11:06:42.035198 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 176/357] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:41 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1523 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:06:42.096020 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-09d1a7f4-7dec-4331-8c87-35e2a72b7b93 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:42.096750 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-09d1a7f4-7dec-4331-8c87-35e2a72b7b93 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:42.103434 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-09d1a7f4-7dec-4331-8c87-35e2a72b7b93 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:42.103892 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-09d1a7f4-7dec-4331-8c87-35e2a72b7b93 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:42.104662 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-09d1a7f4-7dec-4331-8c87-35e2a72b7b93 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:42.105139 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-09d1a7f4-7dec-4331-8c87-35e2a72b7b93 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:42.249939 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-09d1a7f4-7dec-4331-8c87-35e2a72b7b93 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:42.260021 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-09d1a7f4-7dec-4331-8c87-35e2a72b7b93 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1363 microversion: 2.1 time: 0.168313 Jul 19 11:06:42.270748 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 177/358] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:42 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1363 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:06:42.387401 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-b9f44023-5cb8-41be-b4c5-750b5f0ff62d req-e119c3a9-295b-481f-abf7-0c41c84e5286 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "940e8a5c-b365-40c2-8361-fbb677969487", "name": "network-vif-plugged", "status": "completed", "tag": "d928944f-b596-4faa-b70f-efdde2d1f07e"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:06:42.395945 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-b9f44023-5cb8-41be-b4c5-750b5f0ff62d req-e119c3a9-295b-481f-abf7-0c41c84e5286 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:42.396716 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-b9f44023-5cb8-41be-b4c5-750b5f0ff62d req-e119c3a9-295b-481f-abf7-0c41c84e5286 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:42.412533 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-b9f44023-5cb8-41be-b4c5-750b5f0ff62d req-e119c3a9-295b-481f-abf7-0c41c84e5286 service nova] Creating event network-vif-plugged:d928944f-b596-4faa-b70f-efdde2d1f07e for instance 940e8a5c-b365-40c2-8361-fbb677969487 on hv-cinder-maste Jul 19 11:06:42.418046 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-b9f44023-5cb8-41be-b4c5-750b5f0ff62d req-e119c3a9-295b-481f-abf7-0c41c84e5286 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035348 Jul 19 11:06:42.418625 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 178/359] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:06:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:06:43.053070 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-d2fcd62f-46b2-4ee6-804a-a8ba3fb07dff tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:43.053863 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-d2fcd62f-46b2-4ee6-804a-a8ba3fb07dff tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:43.064821 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d2fcd62f-46b2-4ee6-804a-a8ba3fb07dff tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:43.065552 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d2fcd62f-46b2-4ee6-804a-a8ba3fb07dff tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:43.066885 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d2fcd62f-46b2-4ee6-804a-a8ba3fb07dff tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:43.067638 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d2fcd62f-46b2-4ee6-804a-a8ba3fb07dff tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:43.215039 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-d2fcd62f-46b2-4ee6-804a-a8ba3fb07dff tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:43.223159 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-d2fcd62f-46b2-4ee6-804a-a8ba3fb07dff tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1523 microversion: 2.1 time: 0.175403 Jul 19 11:06:43.223953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 179/360] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:43 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1523 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:06:43.278373 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-2f03e86c-c2a4-4a8e-adaa-14d7e825c05b tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:43.278911 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-2f03e86c-c2a4-4a8e-adaa-14d7e825c05b tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:43.285537 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2f03e86c-c2a4-4a8e-adaa-14d7e825c05b tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:43.286096 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2f03e86c-c2a4-4a8e-adaa-14d7e825c05b tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:43.287060 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2f03e86c-c2a4-4a8e-adaa-14d7e825c05b tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:43.287650 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2f03e86c-c2a4-4a8e-adaa-14d7e825c05b tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:43.436658 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-2f03e86c-c2a4-4a8e-adaa-14d7e825c05b tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:43.445507 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-2f03e86c-c2a4-4a8e-adaa-14d7e825c05b tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1363 microversion: 2.1 time: 0.171731 Jul 19 11:06:43.446089 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 180/361] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:43 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1363 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:06:43.577035 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-9c233306-d37d-479f-b3ef-b23be04ae920 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:43.577438 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-9c233306-d37d-479f-b3ef-b23be04ae920 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:43.584849 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9c233306-d37d-479f-b3ef-b23be04ae920 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:43.585304 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9c233306-d37d-479f-b3ef-b23be04ae920 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:43.586084 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9c233306-d37d-479f-b3ef-b23be04ae920 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:43.586523 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9c233306-d37d-479f-b3ef-b23be04ae920 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:43.591520 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:06:43.616071 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-9c233306-d37d-479f-b3ef-b23be04ae920 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372/ips" status: 200 len: 150 microversion: 2.1 time: 0.043647 Jul 19 11:06:43.622149 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 181/362] 10.222.0.33 () {58 vars in 1247 bytes} [Tue Jul 19 11:06:43 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372/ips => generated 150 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:06:44.240760 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-ce5a5d4d-0f7d-4a6c-8d22-4d3a50668050 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:44.241451 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-ce5a5d4d-0f7d-4a6c-8d22-4d3a50668050 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:44.248496 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ce5a5d4d-0f7d-4a6c-8d22-4d3a50668050 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:44.248985 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ce5a5d4d-0f7d-4a6c-8d22-4d3a50668050 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:44.249750 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ce5a5d4d-0f7d-4a6c-8d22-4d3a50668050 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:44.250282 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ce5a5d4d-0f7d-4a6c-8d22-4d3a50668050 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:44.382213 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-ce5a5d4d-0f7d-4a6c-8d22-4d3a50668050 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:44.390620 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-ce5a5d4d-0f7d-4a6c-8d22-4d3a50668050 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1523 microversion: 2.1 time: 0.153865 Jul 19 11:06:44.391193 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 182/363] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:44 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1523 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:06:44.463417 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-8438984e-29ba-4714-acd8-05c308b41b24 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:44.464026 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-8438984e-29ba-4714-acd8-05c308b41b24 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:44.470161 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8438984e-29ba-4714-acd8-05c308b41b24 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:44.470611 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8438984e-29ba-4714-acd8-05c308b41b24 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:44.471369 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8438984e-29ba-4714-acd8-05c308b41b24 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:44.471799 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8438984e-29ba-4714-acd8-05c308b41b24 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:44.596978 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-8438984e-29ba-4714-acd8-05c308b41b24 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:44.604674 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-8438984e-29ba-4714-acd8-05c308b41b24 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1363 microversion: 2.1 time: 0.145144 Jul 19 11:06:44.605250 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 183/364] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:44 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1363 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:06:45.079736 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-e296f822-78b5-40f4-86de-10ca9b6c73cf tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "45d056f5-25af-475a-8cf5-ff31a13487c2"}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:06:45.081784 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-e296f822-78b5-40f4-86de-10ca9b6c73cf tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:45.088882 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e296f822-78b5-40f4-86de-10ca9b6c73cf tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:45.089342 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e296f822-78b5-40f4-86de-10ca9b6c73cf tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:45.090215 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e296f822-78b5-40f4-86de-10ca9b6c73cf tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:45.090654 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e296f822-78b5-40f4-86de-10ca9b6c73cf tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:45.410389 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-e296f822-78b5-40f4-86de-10ca9b6c73cf tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "POST /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.335518 Jul 19 11:06:45.416066 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 184/365] 10.222.0.33 () {60 vars in 1322 bytes} [Tue Jul 19 11:06:45 2022] POST /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372/os-volume_attachments => generated 194 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:06:45.416193 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-0e795d5a-78b2-43fd-89ab-71fcd4d82deb tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-610934161", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-261123105", "security_groups": [{"name": "tempest-secgroup-smoke-1455922164"}], "networks": [{"uuid": "d812e76c-c272-403b-90ca-27996b7fd9fd"}], "block_device_mapping_v2": [{"uuid": "5482b350-3082-4f01-8d65-fb1de19776bc", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:06:45.541805 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.network.neutron [None req-0e795d5a-78b2-43fd-89ab-71fcd4d82deb tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] validate_networks() for [('d812e76c-c272-403b-90ca-27996b7fd9fd', None, None, None, None, None)] {{(pid=117848) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 19 11:06:45.939168 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-0e795d5a-78b2-43fd-89ab-71fcd4d82deb tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Flavor limits 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 19 11:06:45.939628 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-0e795d5a-78b2-43fd-89ab-71fcd4d82deb tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Image limits 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 19 11:06:45.940132 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-0e795d5a-78b2-43fd-89ab-71fcd4d82deb tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Flavor pref 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 19 11:06:45.940577 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-0e795d5a-78b2-43fd-89ab-71fcd4d82deb tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Image pref 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 19 11:06:45.941036 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-0e795d5a-78b2-43fd-89ab-71fcd4d82deb tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 19 11:06:45.951164 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0e795d5a-78b2-43fd-89ab-71fcd4d82deb tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:45.951606 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0e795d5a-78b2-43fd-89ab-71fcd4d82deb tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:45.952428 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0e795d5a-78b2-43fd-89ab-71fcd4d82deb tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:45.952895 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0e795d5a-78b2-43fd-89ab-71fcd4d82deb tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:45.971510 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.quota [None req-0e795d5a-78b2-43fd-89ab-71fcd4d82deb tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Getting quotas for project 375de853371a422f883980296a46ad43. Resources: {'cores', 'instances', 'ram'} {{(pid=117848) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:06:45.975414 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.quota [None req-0e795d5a-78b2-43fd-89ab-71fcd4d82deb tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Getting quotas for user 3ece5061796245a1b2a6bef1d9d6323a and project 375de853371a422f883980296a46ad43. Resources: {'cores', 'instances', 'ram'} {{(pid=117848) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:06:46.032642 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-0e795d5a-78b2-43fd-89ab-71fcd4d82deb tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Going to run 1 instances... {{(pid=117848) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 19 11:06:46.034985 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-0e795d5a-78b2-43fd-89ab-71fcd4d82deb tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='5482b350-3082-4f01-8d65-fb1de19776bc',volume_size=None,volume_type=None)] {{(pid=117848) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 19 11:06:46.184428 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-0e795d5a-78b2-43fd-89ab-71fcd4d82deb tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.772057 Jul 19 11:06:46.185501 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 185/366] 10.222.0.33 () {60 vars in 1146 bytes} [Tue Jul 19 11:06:45 2022] POST /compute/v2.1/servers => generated 406 bytes in 774 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 19 11:06:46.192310 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-3869b869-b98e-49aa-8c69-cbaf148ab075 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:46.192914 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-3869b869-b98e-49aa-8c69-cbaf148ab075 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:46.199751 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3869b869-b98e-49aa-8c69-cbaf148ab075 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:46.200208 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3869b869-b98e-49aa-8c69-cbaf148ab075 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:46.201205 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3869b869-b98e-49aa-8c69-cbaf148ab075 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:46.201644 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3869b869-b98e-49aa-8c69-cbaf148ab075 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:46.405251 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-3869b869-b98e-49aa-8c69-cbaf148ab075 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:46.414833 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-3869b869-b98e-49aa-8c69-cbaf148ab075 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1523 microversion: 2.1 time: 0.227847 Jul 19 11:06:46.425816 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 186/367] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:46 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1523 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:46.432106 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-a5225f57-35ca-4665-bce0-184fcf838696 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:46.432766 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-a5225f57-35ca-4665-bce0-184fcf838696 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:46.439639 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a5225f57-35ca-4665-bce0-184fcf838696 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:46.439927 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a5225f57-35ca-4665-bce0-184fcf838696 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:46.441309 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a5225f57-35ca-4665-bce0-184fcf838696 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:46.442077 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a5225f57-35ca-4665-bce0-184fcf838696 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:46.730154 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-a5225f57-35ca-4665-bce0-184fcf838696 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:46.744535 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-a5225f57-35ca-4665-bce0-184fcf838696 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1549 microversion: 2.1 time: 0.317172 Jul 19 11:06:46.745177 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 187/368] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:46 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1549 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:46.751120 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-14ac9abc-0d11-4a05-bc24-bce626fb2b97 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:46.752218 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-14ac9abc-0d11-4a05-bc24-bce626fb2b97 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:46.820399 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-14ac9abc-0d11-4a05-bc24-bce626fb2b97 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:46.829034 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-14ac9abc-0d11-4a05-bc24-bce626fb2b97 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1087 microversion: 2.1 time: 0.082591 Jul 19 11:06:46.830036 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 188/369] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:46 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1087 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:46.836061 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-018399b5-d3cc-423b-89a7-a10f7a5d77f0 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:46.836766 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-018399b5-d3cc-423b-89a7-a10f7a5d77f0 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:46.844490 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-018399b5-d3cc-423b-89a7-a10f7a5d77f0 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:46.845163 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-018399b5-d3cc-423b-89a7-a10f7a5d77f0 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:46.846258 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-018399b5-d3cc-423b-89a7-a10f7a5d77f0 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:46.846891 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-018399b5-d3cc-423b-89a7-a10f7a5d77f0 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:46.970556 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-018399b5-d3cc-423b-89a7-a10f7a5d77f0 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.138626 Jul 19 11:06:46.971153 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 189/370] 10.222.0.33 () {58 vars in 1274 bytes} [Tue Jul 19 11:06:46 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487/os-interface => generated 284 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:06:47.434008 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-fb2e3d47-92e3-44e4-9a99-aa524f9fffe9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:47.434767 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-fb2e3d47-92e3-44e4-9a99-aa524f9fffe9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:47.464515 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fb2e3d47-92e3-44e4-9a99-aa524f9fffe9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:47.465113 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fb2e3d47-92e3-44e4-9a99-aa524f9fffe9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:47.465899 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fb2e3d47-92e3-44e4-9a99-aa524f9fffe9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:47.466514 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fb2e3d47-92e3-44e4-9a99-aa524f9fffe9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:48.311202 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-fb2e3d47-92e3-44e4-9a99-aa524f9fffe9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:48.327351 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-fb2e3d47-92e3-44e4-9a99-aa524f9fffe9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1523 microversion: 2.1 time: 0.898216 Jul 19 11:06:48.351333 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 190/371] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:47 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1523 bytes in 923 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:48.356777 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-fc23b04d-0442-446c-9b83-832046fcaa46 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:48.357537 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-fc23b04d-0442-446c-9b83-832046fcaa46 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:48.386351 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fc23b04d-0442-446c-9b83-832046fcaa46 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:48.387352 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fc23b04d-0442-446c-9b83-832046fcaa46 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:48.388626 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fc23b04d-0442-446c-9b83-832046fcaa46 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:48.389521 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fc23b04d-0442-446c-9b83-832046fcaa46 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:48.551608 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-fc23b04d-0442-446c-9b83-832046fcaa46 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:48.561847 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-fc23b04d-0442-446c-9b83-832046fcaa46 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1203 microversion: 2.1 time: 0.209158 Jul 19 11:06:48.563093 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 191/372] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:48 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1203 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:49.353356 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-bc35dbaf-cca9-4c7d-aa4a-1d35a0e5120b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:49.354237 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-bc35dbaf-cca9-4c7d-aa4a-1d35a0e5120b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:49.367335 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-bc35dbaf-cca9-4c7d-aa4a-1d35a0e5120b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:49.369436 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-bc35dbaf-cca9-4c7d-aa4a-1d35a0e5120b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:49.370263 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-bc35dbaf-cca9-4c7d-aa4a-1d35a0e5120b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:49.374688 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-bc35dbaf-cca9-4c7d-aa4a-1d35a0e5120b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:49.607115 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-bc35dbaf-cca9-4c7d-aa4a-1d35a0e5120b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:49.622353 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-bc35dbaf-cca9-4c7d-aa4a-1d35a0e5120b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1523 microversion: 2.1 time: 0.276058 Jul 19 11:06:49.623348 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 192/373] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:49 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1523 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:49.637083 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-c37af93a-d15b-4c03-90e2-4bdba2096af7 req-9eb9ce6d-e3c2-496e-8dfc-6246bd99368b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "940e8a5c-b365-40c2-8361-fbb677969487", "tag": "d928944f-b596-4faa-b70f-efdde2d1f07e"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:06:49.646298 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-c37af93a-d15b-4c03-90e2-4bdba2096af7 req-9eb9ce6d-e3c2-496e-8dfc-6246bd99368b service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:49.647492 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-c37af93a-d15b-4c03-90e2-4bdba2096af7 req-9eb9ce6d-e3c2-496e-8dfc-6246bd99368b service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:49.660287 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-c37af93a-d15b-4c03-90e2-4bdba2096af7 req-9eb9ce6d-e3c2-496e-8dfc-6246bd99368b service nova] Creating event network-changed:d928944f-b596-4faa-b70f-efdde2d1f07e for instance 940e8a5c-b365-40c2-8361-fbb677969487 on hv-cinder-maste Jul 19 11:06:49.667409 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-c37af93a-d15b-4c03-90e2-4bdba2096af7 req-9eb9ce6d-e3c2-496e-8dfc-6246bd99368b service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042667 Jul 19 11:06:49.668240 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 193/374] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:06:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:06:49.675137 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-8ac91f22-fadd-4568-868f-4f3d693a9f74 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:49.676110 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-8ac91f22-fadd-4568-868f-4f3d693a9f74 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:49.682819 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8ac91f22-fadd-4568-868f-4f3d693a9f74 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:49.683903 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8ac91f22-fadd-4568-868f-4f3d693a9f74 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:49.684932 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8ac91f22-fadd-4568-868f-4f3d693a9f74 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:49.685635 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8ac91f22-fadd-4568-868f-4f3d693a9f74 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:50.065405 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-8ac91f22-fadd-4568-868f-4f3d693a9f74 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:50.077495 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-8ac91f22-fadd-4568-868f-4f3d693a9f74 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1271 microversion: 2.1 time: 0.407784 Jul 19 11:06:50.078128 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 194/375] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:49 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1271 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:50.160531 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-162cb075-45c4-4a7f-98c2-1927253c2ead req-201c7739-b7c3-4ac9-90f3-a295a5e75ccf service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5b72a14b-bd18-4dfa-a68f-12a5424f471c", "tag": "da843419-4401-4a26-82c2-7930b51aa1b8"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:06:50.192164 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-162cb075-45c4-4a7f-98c2-1927253c2ead req-201c7739-b7c3-4ac9-90f3-a295a5e75ccf service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:50.193674 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-162cb075-45c4-4a7f-98c2-1927253c2ead req-201c7739-b7c3-4ac9-90f3-a295a5e75ccf service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:50.224555 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-162cb075-45c4-4a7f-98c2-1927253c2ead req-201c7739-b7c3-4ac9-90f3-a295a5e75ccf service nova] Creating event network-changed:da843419-4401-4a26-82c2-7930b51aa1b8 for instance 5b72a14b-bd18-4dfa-a68f-12a5424f471c on hv-cinder-maste Jul 19 11:06:50.233118 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-162cb075-45c4-4a7f-98c2-1927253c2ead req-201c7739-b7c3-4ac9-90f3-a295a5e75ccf service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.076536 Jul 19 11:06:50.237548 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 195/376] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:06:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:06:50.506071 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_api:SELECT=44,nova_api:INSERT=3 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:06:50.610383 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=18,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:06:50.640773 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-f844a38d-2610-4bff-9fa5-6410eafcfa1b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:50.642417 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-f844a38d-2610-4bff-9fa5-6410eafcfa1b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:50.660118 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f844a38d-2610-4bff-9fa5-6410eafcfa1b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:50.661477 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f844a38d-2610-4bff-9fa5-6410eafcfa1b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:50.664254 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f844a38d-2610-4bff-9fa5-6410eafcfa1b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:50.665797 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f844a38d-2610-4bff-9fa5-6410eafcfa1b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:51.030981 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-f844a38d-2610-4bff-9fa5-6410eafcfa1b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:51.040500 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-f844a38d-2610-4bff-9fa5-6410eafcfa1b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1523 microversion: 2.1 time: 0.404576 Jul 19 11:06:51.041710 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 196/377] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:50 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1523 bytes in 406 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:51.095149 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-60e22afc-c27a-4de0-b256-c1ac2cfd917c tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:51.095877 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-60e22afc-c27a-4de0-b256-c1ac2cfd917c tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:51.109768 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-60e22afc-c27a-4de0-b256-c1ac2cfd917c tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:51.110290 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-60e22afc-c27a-4de0-b256-c1ac2cfd917c tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:51.111474 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-60e22afc-c27a-4de0-b256-c1ac2cfd917c tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:51.112036 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-60e22afc-c27a-4de0-b256-c1ac2cfd917c tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:51.273305 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-60e22afc-c27a-4de0-b256-c1ac2cfd917c tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:51.291436 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-60e22afc-c27a-4de0-b256-c1ac2cfd917c tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1271 microversion: 2.1 time: 0.200062 Jul 19 11:06:51.292548 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 197/378] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:51 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1271 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:52.057737 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-731e0e75-dd9c-489f-87c0-04eb1502b613 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:52.059184 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-731e0e75-dd9c-489f-87c0-04eb1502b613 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:52.061677 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:06:52.068017 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-731e0e75-dd9c-489f-87c0-04eb1502b613 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:52.068559 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-731e0e75-dd9c-489f-87c0-04eb1502b613 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:52.069418 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-731e0e75-dd9c-489f-87c0-04eb1502b613 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:52.070026 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-731e0e75-dd9c-489f-87c0-04eb1502b613 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:52.213903 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-731e0e75-dd9c-489f-87c0-04eb1502b613 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:52.221682 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-731e0e75-dd9c-489f-87c0-04eb1502b613 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1523 microversion: 2.1 time: 0.167850 Jul 19 11:06:52.222322 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 198/379] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:52 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1523 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:06:52.308937 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-dc0e3250-92ed-47b4-8999-e1fa9630d681 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:52.309464 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-dc0e3250-92ed-47b4-8999-e1fa9630d681 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:52.315658 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dc0e3250-92ed-47b4-8999-e1fa9630d681 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:52.316104 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dc0e3250-92ed-47b4-8999-e1fa9630d681 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:52.317057 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dc0e3250-92ed-47b4-8999-e1fa9630d681 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:52.317661 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dc0e3250-92ed-47b4-8999-e1fa9630d681 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:52.478886 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-dc0e3250-92ed-47b4-8999-e1fa9630d681 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:52.486810 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-dc0e3250-92ed-47b4-8999-e1fa9630d681 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1271 microversion: 2.1 time: 0.181855 Jul 19 11:06:52.497004 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 199/380] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:52 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1271 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:06:52.583960 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-2f62f16e-1bae-40fc-90b8-e08dcf840689 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "POST /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326/action" status: 202 len: 0 microversion: 2.20 time: 12.087619 Jul 19 11:06:52.588954 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 182/381] 10.222.0.33 () {62 vars in 1318 bytes} [Tue Jul 19 11:06:40 2022] POST /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326/action => generated 0 bytes in 12093 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 19 11:06:52.594155 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-bb86b0a6-0476-44de-b86a-f78275a777d3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:52.594846 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-bb86b0a6-0476-44de-b86a-f78275a777d3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:52.601447 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-bb86b0a6-0476-44de-b86a-f78275a777d3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:52.601953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-bb86b0a6-0476-44de-b86a-f78275a777d3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:52.603574 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-bb86b0a6-0476-44de-b86a-f78275a777d3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:52.604089 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-bb86b0a6-0476-44de-b86a-f78275a777d3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:52.609456 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:06:52.742564 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-bb86b0a6-0476-44de-b86a-f78275a777d3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:52.751182 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-bb86b0a6-0476-44de-b86a-f78275a777d3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:52.752721 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-bb86b0a6-0476-44de-b86a-f78275a777d3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:52.753882 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-bb86b0a6-0476-44de-b86a-f78275a777d3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1722 microversion: 2.20 time: 0.163732 Jul 19 11:06:52.754411 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 200/382] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:06:52 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1722 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:06:53.238421 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-9d7d77c0-d5fa-471a-9b08-a80c325bfda3 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:53.239106 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-9d7d77c0-d5fa-471a-9b08-a80c325bfda3 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:53.245687 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9d7d77c0-d5fa-471a-9b08-a80c325bfda3 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:53.246128 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9d7d77c0-d5fa-471a-9b08-a80c325bfda3 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:53.246882 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9d7d77c0-d5fa-471a-9b08-a80c325bfda3 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:53.247309 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9d7d77c0-d5fa-471a-9b08-a80c325bfda3 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:53.375753 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-9d7d77c0-d5fa-471a-9b08-a80c325bfda3 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:53.383128 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-9d7d77c0-d5fa-471a-9b08-a80c325bfda3 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1523 microversion: 2.1 time: 0.148983 Jul 19 11:06:53.383687 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 183/383] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:53 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1523 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:53.503312 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-2c0b8546-5172-4037-9c4c-148730994c16 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:53.503914 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-2c0b8546-5172-4037-9c4c-148730994c16 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:53.511452 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2c0b8546-5172-4037-9c4c-148730994c16 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:53.511895 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2c0b8546-5172-4037-9c4c-148730994c16 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:53.512645 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2c0b8546-5172-4037-9c4c-148730994c16 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:53.513121 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2c0b8546-5172-4037-9c4c-148730994c16 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:53.651539 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-2c0b8546-5172-4037-9c4c-148730994c16 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:53.659000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-2c0b8546-5172-4037-9c4c-148730994c16 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1271 microversion: 2.1 time: 0.159454 Jul 19 11:06:53.659564 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 201/384] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:53 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1271 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:53.772855 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-10e8a7d2-b854-4184-8799-c31cd342ab1f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:53.773454 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-10e8a7d2-b854-4184-8799-c31cd342ab1f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:53.779439 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-10e8a7d2-b854-4184-8799-c31cd342ab1f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:53.779876 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-10e8a7d2-b854-4184-8799-c31cd342ab1f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:53.780641 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-10e8a7d2-b854-4184-8799-c31cd342ab1f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:53.781094 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-10e8a7d2-b854-4184-8799-c31cd342ab1f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:53.907136 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-10e8a7d2-b854-4184-8799-c31cd342ab1f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:53.915810 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-10e8a7d2-b854-4184-8799-c31cd342ab1f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:53.917394 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-10e8a7d2-b854-4184-8799-c31cd342ab1f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:53.918531 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-10e8a7d2-b854-4184-8799-c31cd342ab1f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1722 microversion: 2.20 time: 0.149744 Jul 19 11:06:53.919081 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 184/385] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:06:53 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1722 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:06:54.400385 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-8213fae4-0fe0-4618-b84e-5eae1f59ba09 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:54.401020 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-8213fae4-0fe0-4618-b84e-5eae1f59ba09 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:54.407132 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8213fae4-0fe0-4618-b84e-5eae1f59ba09 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:54.407588 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8213fae4-0fe0-4618-b84e-5eae1f59ba09 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:54.408355 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8213fae4-0fe0-4618-b84e-5eae1f59ba09 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:54.408826 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8213fae4-0fe0-4618-b84e-5eae1f59ba09 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:54.572573 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-8213fae4-0fe0-4618-b84e-5eae1f59ba09 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:54.583775 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-8213fae4-0fe0-4618-b84e-5eae1f59ba09 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1523 microversion: 2.1 time: 0.187687 Jul 19 11:06:54.585016 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 202/386] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:54 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1523 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:54.676494 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-fda7bcd9-8665-4b84-a1ec-4f03806177fd tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:54.677135 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-fda7bcd9-8665-4b84-a1ec-4f03806177fd tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:54.684007 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fda7bcd9-8665-4b84-a1ec-4f03806177fd tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:54.684724 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fda7bcd9-8665-4b84-a1ec-4f03806177fd tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:54.685826 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fda7bcd9-8665-4b84-a1ec-4f03806177fd tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:54.686427 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fda7bcd9-8665-4b84-a1ec-4f03806177fd tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:54.835107 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-fda7bcd9-8665-4b84-a1ec-4f03806177fd tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:54.844127 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-fda7bcd9-8665-4b84-a1ec-4f03806177fd tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1271 microversion: 2.1 time: 0.172428 Jul 19 11:06:54.844781 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 185/387] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:54 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1271 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:54.951346 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-a9bac54a-d662-4b03-9c5e-51759b7ec258 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:54.951971 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-a9bac54a-d662-4b03-9c5e-51759b7ec258 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:54.962288 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a9bac54a-d662-4b03-9c5e-51759b7ec258 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:54.963422 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a9bac54a-d662-4b03-9c5e-51759b7ec258 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:54.964219 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a9bac54a-d662-4b03-9c5e-51759b7ec258 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:54.964660 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a9bac54a-d662-4b03-9c5e-51759b7ec258 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:55.158953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-a9bac54a-d662-4b03-9c5e-51759b7ec258 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:55.172103 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-a9bac54a-d662-4b03-9c5e-51759b7ec258 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:55.174127 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-a9bac54a-d662-4b03-9c5e-51759b7ec258 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:55.175956 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-a9bac54a-d662-4b03-9c5e-51759b7ec258 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1722 microversion: 2.20 time: 0.228526 Jul 19 11:06:55.177028 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 203/388] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:06:54 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1722 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:06:55.601802 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-77454b0e-d013-403d-8d9c-fe467857b93d tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:55.603301 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-77454b0e-d013-403d-8d9c-fe467857b93d tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:55.615493 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-77454b0e-d013-403d-8d9c-fe467857b93d tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:55.618359 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-77454b0e-d013-403d-8d9c-fe467857b93d tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:55.619152 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-77454b0e-d013-403d-8d9c-fe467857b93d tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:55.619591 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-77454b0e-d013-403d-8d9c-fe467857b93d tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:55.781261 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-77454b0e-d013-403d-8d9c-fe467857b93d tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:55.789053 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-77454b0e-d013-403d-8d9c-fe467857b93d tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1523 microversion: 2.1 time: 0.191026 Jul 19 11:06:55.800090 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 186/389] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:55 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1523 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:55.862388 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-2fee95e9-1506-492a-b4ff-5fffd7e2f869 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:55.863632 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-2fee95e9-1506-492a-b4ff-5fffd7e2f869 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:55.870877 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2fee95e9-1506-492a-b4ff-5fffd7e2f869 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:55.871355 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2fee95e9-1506-492a-b4ff-5fffd7e2f869 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:55.872118 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2fee95e9-1506-492a-b4ff-5fffd7e2f869 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:55.872543 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2fee95e9-1506-492a-b4ff-5fffd7e2f869 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:55.960652 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell0:SELECT=2 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:06:56.017588 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-2fee95e9-1506-492a-b4ff-5fffd7e2f869 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:56.025408 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-2fee95e9-1506-492a-b4ff-5fffd7e2f869 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1271 microversion: 2.1 time: 0.166791 Jul 19 11:06:56.029132 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 204/390] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:55 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1271 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:56.194716 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-74255393-35ab-4ea6-b898-67fe64fe43f8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:56.196073 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-74255393-35ab-4ea6-b898-67fe64fe43f8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:56.202518 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-74255393-35ab-4ea6-b898-67fe64fe43f8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:56.202968 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-74255393-35ab-4ea6-b898-67fe64fe43f8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:56.203733 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-74255393-35ab-4ea6-b898-67fe64fe43f8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:56.204370 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-74255393-35ab-4ea6-b898-67fe64fe43f8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:56.370613 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-74255393-35ab-4ea6-b898-67fe64fe43f8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:56.378767 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-74255393-35ab-4ea6-b898-67fe64fe43f8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:56.380273 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-74255393-35ab-4ea6-b898-67fe64fe43f8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:56.381486 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-74255393-35ab-4ea6-b898-67fe64fe43f8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1722 microversion: 2.20 time: 0.190780 Jul 19 11:06:56.382227 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 187/391] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:06:56 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1722 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:06:56.809003 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-5c466f4b-32f5-4d70-a85d-a1a63db0d454 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:56.809904 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-5c466f4b-32f5-4d70-a85d-a1a63db0d454 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:56.817589 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5c466f4b-32f5-4d70-a85d-a1a63db0d454 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:56.818046 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5c466f4b-32f5-4d70-a85d-a1a63db0d454 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:56.818823 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5c466f4b-32f5-4d70-a85d-a1a63db0d454 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:56.819378 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5c466f4b-32f5-4d70-a85d-a1a63db0d454 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:56.825843 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell0:SELECT=8 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:06:56.963547 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-5c466f4b-32f5-4d70-a85d-a1a63db0d454 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:56.971457 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-5c466f4b-32f5-4d70-a85d-a1a63db0d454 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1523 microversion: 2.1 time: 0.166759 Jul 19 11:06:56.972120 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 205/392] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:56 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1523 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:57.046051 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-f008e195-da93-4c90-aa95-0a6fefec1b51 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:57.046673 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-f008e195-da93-4c90-aa95-0a6fefec1b51 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:57.064368 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f008e195-da93-4c90-aa95-0a6fefec1b51 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:57.064855 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f008e195-da93-4c90-aa95-0a6fefec1b51 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:57.065643 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f008e195-da93-4c90-aa95-0a6fefec1b51 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:57.066095 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f008e195-da93-4c90-aa95-0a6fefec1b51 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:57.210672 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-f008e195-da93-4c90-aa95-0a6fefec1b51 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:57.221336 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-f008e195-da93-4c90-aa95-0a6fefec1b51 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1271 microversion: 2.1 time: 0.183014 Jul 19 11:06:57.222222 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 188/393] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:57 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1271 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:57.399857 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-969e86ef-a561-49c1-ab9c-2daf11e8bb6e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:57.400830 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-969e86ef-a561-49c1-ab9c-2daf11e8bb6e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:57.407290 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-969e86ef-a561-49c1-ab9c-2daf11e8bb6e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:57.407932 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-969e86ef-a561-49c1-ab9c-2daf11e8bb6e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:57.408979 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-969e86ef-a561-49c1-ab9c-2daf11e8bb6e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:57.409587 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-969e86ef-a561-49c1-ab9c-2daf11e8bb6e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:57.562798 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-969e86ef-a561-49c1-ab9c-2daf11e8bb6e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:57.572022 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-969e86ef-a561-49c1-ab9c-2daf11e8bb6e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:57.573910 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-969e86ef-a561-49c1-ab9c-2daf11e8bb6e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:57.574903 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-969e86ef-a561-49c1-ab9c-2daf11e8bb6e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1722 microversion: 2.20 time: 0.179494 Jul 19 11:06:57.576975 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 206/394] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:06:57 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1722 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:06:57.997574 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-0eec8350-1773-4449-a845-fcea020a0d1e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:57.998473 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-0eec8350-1773-4449-a845-fcea020a0d1e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:58.005547 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0eec8350-1773-4449-a845-fcea020a0d1e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:58.006959 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0eec8350-1773-4449-a845-fcea020a0d1e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:58.009585 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0eec8350-1773-4449-a845-fcea020a0d1e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:58.009585 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0eec8350-1773-4449-a845-fcea020a0d1e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:58.173838 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-0eec8350-1773-4449-a845-fcea020a0d1e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:58.182883 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-0eec8350-1773-4449-a845-fcea020a0d1e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1523 microversion: 2.1 time: 0.189222 Jul 19 11:06:58.183462 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 189/395] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:57 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1523 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:58.239114 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-b07bdc12-d6d4-4f28-bf59-7e5e3dcfd95f tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:58.239672 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-b07bdc12-d6d4-4f28-bf59-7e5e3dcfd95f tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:58.247512 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b07bdc12-d6d4-4f28-bf59-7e5e3dcfd95f tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:58.248315 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b07bdc12-d6d4-4f28-bf59-7e5e3dcfd95f tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:58.249244 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b07bdc12-d6d4-4f28-bf59-7e5e3dcfd95f tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:58.249754 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b07bdc12-d6d4-4f28-bf59-7e5e3dcfd95f tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:58.415943 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-b07bdc12-d6d4-4f28-bf59-7e5e3dcfd95f tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:58.427089 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-b07bdc12-d6d4-4f28-bf59-7e5e3dcfd95f tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1271 microversion: 2.1 time: 0.192188 Jul 19 11:06:58.437278 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 207/396] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:58 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1271 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:58.594030 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-c21ba1a2-11a1-4dd9-bf40-d2508d6e68e5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:58.594622 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-c21ba1a2-11a1-4dd9-bf40-d2508d6e68e5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:58.600855 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c21ba1a2-11a1-4dd9-bf40-d2508d6e68e5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:58.601306 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c21ba1a2-11a1-4dd9-bf40-d2508d6e68e5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:58.605555 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c21ba1a2-11a1-4dd9-bf40-d2508d6e68e5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:58.606057 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c21ba1a2-11a1-4dd9-bf40-d2508d6e68e5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:58.762815 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-c21ba1a2-11a1-4dd9-bf40-d2508d6e68e5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:58.775882 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-c21ba1a2-11a1-4dd9-bf40-d2508d6e68e5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:58.778128 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-c21ba1a2-11a1-4dd9-bf40-d2508d6e68e5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:58.780505 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-c21ba1a2-11a1-4dd9-bf40-d2508d6e68e5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1722 microversion: 2.20 time: 0.190545 Jul 19 11:06:58.781621 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 190/397] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:06:58 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1722 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:06:59.200500 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-b0d437c2-1f9f-425f-add6-4f246bf4ffd4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:59.201068 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-b0d437c2-1f9f-425f-add6-4f246bf4ffd4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:59.207022 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b0d437c2-1f9f-425f-add6-4f246bf4ffd4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:59.207474 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b0d437c2-1f9f-425f-add6-4f246bf4ffd4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:59.208234 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b0d437c2-1f9f-425f-add6-4f246bf4ffd4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:59.208673 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b0d437c2-1f9f-425f-add6-4f246bf4ffd4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:59.351237 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-b0d437c2-1f9f-425f-add6-4f246bf4ffd4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:59.359284 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-b0d437c2-1f9f-425f-add6-4f246bf4ffd4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1523 microversion: 2.1 time: 0.163417 Jul 19 11:06:59.359908 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 208/398] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:59 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1523 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:59.446744 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-d25bb53b-e826-4b80-adc6-3579a639e39c tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:59.448277 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-d25bb53b-e826-4b80-adc6-3579a639e39c tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:59.455134 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d25bb53b-e826-4b80-adc6-3579a639e39c tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:59.455905 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d25bb53b-e826-4b80-adc6-3579a639e39c tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:59.456994 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d25bb53b-e826-4b80-adc6-3579a639e39c tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:59.457657 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d25bb53b-e826-4b80-adc6-3579a639e39c tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:59.668869 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-d25bb53b-e826-4b80-adc6-3579a639e39c tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:59.679315 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-d25bb53b-e826-4b80-adc6-3579a639e39c tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1271 microversion: 2.1 time: 0.237383 Jul 19 11:06:59.679911 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 191/399] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:06:59 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1271 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:06:59.802325 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-ebe81dcb-d03a-4148-926b-1f557a247c19 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:06:59.809057 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-ebe81dcb-d03a-4148-926b-1f557a247c19 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:06:59.815882 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ebe81dcb-d03a-4148-926b-1f557a247c19 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:59.816616 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ebe81dcb-d03a-4148-926b-1f557a247c19 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:59.817589 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ebe81dcb-d03a-4148-926b-1f557a247c19 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:06:59.818028 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ebe81dcb-d03a-4148-926b-1f557a247c19 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:06:59.959221 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-ebe81dcb-d03a-4148-926b-1f557a247c19 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:59.968996 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-ebe81dcb-d03a-4148-926b-1f557a247c19 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:59.970911 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-ebe81dcb-d03a-4148-926b-1f557a247c19 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:06:59.972093 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-ebe81dcb-d03a-4148-926b-1f557a247c19 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1722 microversion: 2.20 time: 0.176971 Jul 19 11:06:59.972688 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 209/400] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:06:59 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1722 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:00.376259 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-9629e9e2-1e64-42c5-a233-8e89e89af6fa tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:00.377089 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-9629e9e2-1e64-42c5-a233-8e89e89af6fa tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:00.383855 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9629e9e2-1e64-42c5-a233-8e89e89af6fa tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:00.384531 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9629e9e2-1e64-42c5-a233-8e89e89af6fa tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:00.385324 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9629e9e2-1e64-42c5-a233-8e89e89af6fa tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:00.385826 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9629e9e2-1e64-42c5-a233-8e89e89af6fa tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:00.529835 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-9629e9e2-1e64-42c5-a233-8e89e89af6fa tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:00.537754 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-9629e9e2-1e64-42c5-a233-8e89e89af6fa tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1523 microversion: 2.1 time: 0.165491 Jul 19 11:07:00.538563 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 192/401] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:00 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1523 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:00.696913 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-3ccbd92d-82d7-4cde-ab4b-2026ef5d753d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:00.697629 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-3ccbd92d-82d7-4cde-ab4b-2026ef5d753d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:00.704472 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3ccbd92d-82d7-4cde-ab4b-2026ef5d753d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:00.705241 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3ccbd92d-82d7-4cde-ab4b-2026ef5d753d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:00.706125 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3ccbd92d-82d7-4cde-ab4b-2026ef5d753d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:00.706766 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3ccbd92d-82d7-4cde-ab4b-2026ef5d753d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:00.877397 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-3ccbd92d-82d7-4cde-ab4b-2026ef5d753d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:00.896109 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-3ccbd92d-82d7-4cde-ab4b-2026ef5d753d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1271 microversion: 2.1 time: 0.203118 Jul 19 11:07:00.897072 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 210/402] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:00 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1271 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:00.991257 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-a5cf290a-54e4-4c97-9912-6f4dcfd3d17e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:00.991847 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-a5cf290a-54e4-4c97-9912-6f4dcfd3d17e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:00.999320 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a5cf290a-54e4-4c97-9912-6f4dcfd3d17e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:01.000065 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a5cf290a-54e4-4c97-9912-6f4dcfd3d17e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:01.000903 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a5cf290a-54e4-4c97-9912-6f4dcfd3d17e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:01.001347 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a5cf290a-54e4-4c97-9912-6f4dcfd3d17e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:01.151687 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-a5cf290a-54e4-4c97-9912-6f4dcfd3d17e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:01.159853 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-a5cf290a-54e4-4c97-9912-6f4dcfd3d17e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:01.161407 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-a5cf290a-54e4-4c97-9912-6f4dcfd3d17e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:01.162612 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-a5cf290a-54e4-4c97-9912-6f4dcfd3d17e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1722 microversion: 2.20 time: 0.175467 Jul 19 11:07:01.163485 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 193/403] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:00 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1722 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:01.562554 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-87fd0de1-7e1e-49d8-9517-e4253ba54a82 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:01.563326 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-87fd0de1-7e1e-49d8-9517-e4253ba54a82 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:01.571568 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-87fd0de1-7e1e-49d8-9517-e4253ba54a82 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:01.572065 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-87fd0de1-7e1e-49d8-9517-e4253ba54a82 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:01.574154 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-87fd0de1-7e1e-49d8-9517-e4253ba54a82 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:01.574154 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-87fd0de1-7e1e-49d8-9517-e4253ba54a82 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:01.728641 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-87fd0de1-7e1e-49d8-9517-e4253ba54a82 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:01.736497 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-87fd0de1-7e1e-49d8-9517-e4253ba54a82 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1523 microversion: 2.1 time: 0.182356 Jul 19 11:07:01.737213 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 211/404] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:01 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1523 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:01.914082 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-b3a39628-0d81-43d1-a316-cae76be972d6 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:01.914984 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-b3a39628-0d81-43d1-a316-cae76be972d6 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:01.921656 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b3a39628-0d81-43d1-a316-cae76be972d6 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:01.922332 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b3a39628-0d81-43d1-a316-cae76be972d6 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:01.923296 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b3a39628-0d81-43d1-a316-cae76be972d6 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:01.923889 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b3a39628-0d81-43d1-a316-cae76be972d6 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:02.073404 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b3a39628-0d81-43d1-a316-cae76be972d6 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:02.081173 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-b3a39628-0d81-43d1-a316-cae76be972d6 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1271 microversion: 2.1 time: 0.171001 Jul 19 11:07:02.081971 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 194/405] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:01 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1271 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:02.182734 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-36c3795c-b4c6-4f2d-a09a-e044fb29cd4c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:02.183464 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-36c3795c-b4c6-4f2d-a09a-e044fb29cd4c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:02.194370 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-36c3795c-b4c6-4f2d-a09a-e044fb29cd4c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:02.197280 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-36c3795c-b4c6-4f2d-a09a-e044fb29cd4c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:02.198064 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-36c3795c-b4c6-4f2d-a09a-e044fb29cd4c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:02.198498 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-36c3795c-b4c6-4f2d-a09a-e044fb29cd4c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:02.354820 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-36c3795c-b4c6-4f2d-a09a-e044fb29cd4c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:02.366944 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-36c3795c-b4c6-4f2d-a09a-e044fb29cd4c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:02.369321 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-36c3795c-b4c6-4f2d-a09a-e044fb29cd4c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:02.370972 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-36c3795c-b4c6-4f2d-a09a-e044fb29cd4c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1722 microversion: 2.20 time: 0.192572 Jul 19 11:07:02.371746 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 212/406] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:02 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1722 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:02.757160 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-a67f9342-e203-4cc7-a298-531ad4865ced tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:02.758007 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-a67f9342-e203-4cc7-a298-531ad4865ced tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:02.766112 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a67f9342-e203-4cc7-a298-531ad4865ced tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:02.766833 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a67f9342-e203-4cc7-a298-531ad4865ced tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:02.767831 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a67f9342-e203-4cc7-a298-531ad4865ced tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:02.768539 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a67f9342-e203-4cc7-a298-531ad4865ced tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:02.936653 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-a67f9342-e203-4cc7-a298-531ad4865ced tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:02.946517 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-a67f9342-e203-4cc7-a298-531ad4865ced tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1523 microversion: 2.1 time: 0.195540 Jul 19 11:07:02.956681 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 195/407] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:02 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1523 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:03.098394 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-6e044992-c201-4a33-9fa5-3d817b08b7ee tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:03.099648 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-6e044992-c201-4a33-9fa5-3d817b08b7ee tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:03.106977 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6e044992-c201-4a33-9fa5-3d817b08b7ee tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:03.107429 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6e044992-c201-4a33-9fa5-3d817b08b7ee tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:03.108187 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6e044992-c201-4a33-9fa5-3d817b08b7ee tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:03.108613 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6e044992-c201-4a33-9fa5-3d817b08b7ee tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:03.296354 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-6e044992-c201-4a33-9fa5-3d817b08b7ee tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:03.309927 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-6e044992-c201-4a33-9fa5-3d817b08b7ee tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1271 microversion: 2.1 time: 0.215350 Jul 19 11:07:03.311251 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 213/408] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:03 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1271 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:03.389932 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-95ff0147-d2ad-439f-bc2a-9adcc47a7b94 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:03.390537 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-95ff0147-d2ad-439f-bc2a-9adcc47a7b94 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:03.397956 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-95ff0147-d2ad-439f-bc2a-9adcc47a7b94 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:03.398697 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-95ff0147-d2ad-439f-bc2a-9adcc47a7b94 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:03.400516 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-95ff0147-d2ad-439f-bc2a-9adcc47a7b94 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:03.401233 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-95ff0147-d2ad-439f-bc2a-9adcc47a7b94 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:03.553523 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-95ff0147-d2ad-439f-bc2a-9adcc47a7b94 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:03.565641 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-95ff0147-d2ad-439f-bc2a-9adcc47a7b94 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:03.567445 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-95ff0147-d2ad-439f-bc2a-9adcc47a7b94 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:03.569160 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-95ff0147-d2ad-439f-bc2a-9adcc47a7b94 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1722 microversion: 2.20 time: 0.183110 Jul 19 11:07:03.570851 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 196/409] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:03 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1722 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:03.878936 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-3b97d476-a41c-4bff-9146-d675dd63f20d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Action: 'create', calling method: .version_select at 0x7f04aa4d2310>, body: {"keypair": {"name": "tempest-keypair-40836903", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJnSrrJrAkkUkzdwSDLE/ugigHOho2Lv2sv4ze3vQr4WwCd0NPYnkBzEL3nSgrzPIYPfVCWnentQ+EgVIOYHsamcyevNDKobjaA2IUGz4St0aQXQMNMF1tRnVFtjolji8g=="}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:07:03.895066 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.quota [None req-3b97d476-a41c-4bff-9146-d675dd63f20d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Getting quotas for project ef534f1d06ea415d98857ec7f9e5fb97. Resources: {'key_pairs'} {{(pid=117848) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:07:03.900002 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.quota [None req-3b97d476-a41c-4bff-9146-d675dd63f20d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Getting quotas for user b8f2a78f3a3e4b8e8937e3bc39524690 and project ef534f1d06ea415d98857ec7f9e5fb97. Resources: {'key_pairs'} {{(pid=117848) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:07:03.931139 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-3b97d476-a41c-4bff-9146-d675dd63f20d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "POST /compute/v2.1/os-keypairs" status: 200 len: 384 microversion: 2.1 time: 0.056013 Jul 19 11:07:03.932035 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 214/410] 10.222.0.33 () {60 vars in 1158 bytes} [Tue Jul 19 11:07:03 2022] POST /compute/v2.1/os-keypairs => generated 384 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:07:03.977822 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-16f114f8-e836-458c-87a6-9daab001273c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:03.977822 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-16f114f8-e836-458c-87a6-9daab001273c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:03.991948 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-16f114f8-e836-458c-87a6-9daab001273c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:03.992756 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-16f114f8-e836-458c-87a6-9daab001273c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:03.993697 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-16f114f8-e836-458c-87a6-9daab001273c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:03.994339 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-16f114f8-e836-458c-87a6-9daab001273c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:04.213165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-16f114f8-e836-458c-87a6-9daab001273c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:04.227629 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-16f114f8-e836-458c-87a6-9daab001273c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1523 microversion: 2.1 time: 0.257649 Jul 19 11:07:04.228393 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 197/411] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:03 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1523 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:04.328990 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-d0e7a7ba-6786-4486-83c2-347dbea5a237 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:04.329600 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-d0e7a7ba-6786-4486-83c2-347dbea5a237 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:04.337113 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d0e7a7ba-6786-4486-83c2-347dbea5a237 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:04.337560 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d0e7a7ba-6786-4486-83c2-347dbea5a237 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:04.338320 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d0e7a7ba-6786-4486-83c2-347dbea5a237 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:04.338749 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d0e7a7ba-6786-4486-83c2-347dbea5a237 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:04.484994 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-d0e7a7ba-6786-4486-83c2-347dbea5a237 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:04.493218 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-d0e7a7ba-6786-4486-83c2-347dbea5a237 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1271 microversion: 2.1 time: 0.169123 Jul 19 11:07:04.493789 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 215/412] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:04 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1271 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:07:04.587774 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-80b1e1e8-1479-4d47-8989-7ba257afd53f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:04.588537 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-80b1e1e8-1479-4d47-8989-7ba257afd53f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:04.594983 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-80b1e1e8-1479-4d47-8989-7ba257afd53f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:04.595671 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-80b1e1e8-1479-4d47-8989-7ba257afd53f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:04.596449 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-80b1e1e8-1479-4d47-8989-7ba257afd53f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:04.596917 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-80b1e1e8-1479-4d47-8989-7ba257afd53f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:05.109351 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-80b1e1e8-1479-4d47-8989-7ba257afd53f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:05.119169 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-80b1e1e8-1479-4d47-8989-7ba257afd53f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:05.120672 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-80b1e1e8-1479-4d47-8989-7ba257afd53f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:05.124542 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-80b1e1e8-1479-4d47-8989-7ba257afd53f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1722 microversion: 2.20 time: 0.540955 Jul 19 11:07:05.125138 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 198/413] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:04 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1722 bytes in 542 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:05.245945 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-4d2e4a9a-2be8-4835-9fd4-8c704253a53c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:05.246881 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-4d2e4a9a-2be8-4835-9fd4-8c704253a53c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:05.257788 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4d2e4a9a-2be8-4835-9fd4-8c704253a53c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:05.267308 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4d2e4a9a-2be8-4835-9fd4-8c704253a53c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:05.268754 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4d2e4a9a-2be8-4835-9fd4-8c704253a53c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:05.269355 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4d2e4a9a-2be8-4835-9fd4-8c704253a53c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:05.468766 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-4d2e4a9a-2be8-4835-9fd4-8c704253a53c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:05.480631 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-4d2e4a9a-2be8-4835-9fd4-8c704253a53c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1523 microversion: 2.1 time: 0.239262 Jul 19 11:07:05.481758 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 216/414] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:05 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1523 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:05.510813 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-5d823961-de46-4879-9b90-b0c645ba69b7 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:05.511413 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-5d823961-de46-4879-9b90-b0c645ba69b7 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:05.520907 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5d823961-de46-4879-9b90-b0c645ba69b7 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:05.525322 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5d823961-de46-4879-9b90-b0c645ba69b7 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:05.526341 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5d823961-de46-4879-9b90-b0c645ba69b7 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:05.526965 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5d823961-de46-4879-9b90-b0c645ba69b7 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:05.549488 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=100 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:07:05.792377 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-5d823961-de46-4879-9b90-b0c645ba69b7 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:05.802378 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-5d823961-de46-4879-9b90-b0c645ba69b7 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1259 microversion: 2.1 time: 0.295935 Jul 19 11:07:05.802972 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 199/415] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:05 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1259 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:06.012727 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-5521d2f0-d833-4707-9d3d-7e9fd1357fe5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-520432940", "imageRef": "3085883e-2289-4314-aa47-9c41d4368b02", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "0b1fabad-b81a-4ee7-9fd9-62393ba0d37f"}], "security_groups": [{"name": "tempest-securitygroup--803536172"}], "key_name": "tempest-keypair-40836903", "user_data": "}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:07:06.112116 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.network.neutron [None req-5521d2f0-d833-4707-9d3d-7e9fd1357fe5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] validate_networks() for [('0b1fabad-b81a-4ee7-9fd9-62393ba0d37f', None, None, None, None, None)] {{(pid=117848) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 19 11:07:06.143463 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-ba7b9f55-d753-4f86-8a40-ff1a12b06623 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:06.144094 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-ba7b9f55-d753-4f86-8a40-ff1a12b06623 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:06.149970 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ba7b9f55-d753-4f86-8a40-ff1a12b06623 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:06.151102 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ba7b9f55-d753-4f86-8a40-ff1a12b06623 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:06.152051 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ba7b9f55-d753-4f86-8a40-ff1a12b06623 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:06.152826 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ba7b9f55-d753-4f86-8a40-ff1a12b06623 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:06.287483 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-ba7b9f55-d753-4f86-8a40-ff1a12b06623 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:06.296058 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-ba7b9f55-d753-4f86-8a40-ff1a12b06623 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:06.297702 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-ba7b9f55-d753-4f86-8a40-ff1a12b06623 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:06.298850 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-ba7b9f55-d753-4f86-8a40-ff1a12b06623 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1722 microversion: 2.20 time: 0.159484 Jul 19 11:07:06.299406 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 200/416] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:06 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1722 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:07:06.496836 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-5521d2f0-d833-4707-9d3d-7e9fd1357fe5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Flavor limits 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 19 11:07:06.497267 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-5521d2f0-d833-4707-9d3d-7e9fd1357fe5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Image limits 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 19 11:07:06.497960 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-5521d2f0-d833-4707-9d3d-7e9fd1357fe5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Flavor pref 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 19 11:07:06.498364 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-5521d2f0-d833-4707-9d3d-7e9fd1357fe5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Image pref 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 19 11:07:06.498769 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-5521d2f0-d833-4707-9d3d-7e9fd1357fe5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 19 11:07:06.500804 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-b2075aa5-97ee-448e-aa18-ab4f64ca55b7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:06.501480 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-b2075aa5-97ee-448e-aa18-ab4f64ca55b7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:06.507879 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b2075aa5-97ee-448e-aa18-ab4f64ca55b7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:06.508223 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5521d2f0-d833-4707-9d3d-7e9fd1357fe5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:06.508469 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b2075aa5-97ee-448e-aa18-ab4f64ca55b7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:06.508914 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5521d2f0-d833-4707-9d3d-7e9fd1357fe5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:06.509405 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b2075aa5-97ee-448e-aa18-ab4f64ca55b7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:06.509669 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5521d2f0-d833-4707-9d3d-7e9fd1357fe5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:06.509870 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b2075aa5-97ee-448e-aa18-ab4f64ca55b7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:06.510219 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5521d2f0-d833-4707-9d3d-7e9fd1357fe5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:06.527097 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.quota [None req-5521d2f0-d833-4707-9d3d-7e9fd1357fe5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Getting quotas for project ef534f1d06ea415d98857ec7f9e5fb97. Resources: {'cores', 'instances', 'ram'} {{(pid=117848) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:07:06.531260 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.quota [None req-5521d2f0-d833-4707-9d3d-7e9fd1357fe5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Getting quotas for user b8f2a78f3a3e4b8e8937e3bc39524690 and project ef534f1d06ea415d98857ec7f9e5fb97. Resources: {'cores', 'instances', 'ram'} {{(pid=117848) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:07:06.543222 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-5521d2f0-d833-4707-9d3d-7e9fd1357fe5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Going to run 1 instances... {{(pid=117848) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 19 11:07:06.545518 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-5521d2f0-d833-4707-9d3d-7e9fd1357fe5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 988f14d7-6354-4388-a51c-b4df9749c671] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='3085883e-2289-4314-aa47-9c41d4368b02',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117848) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 19 11:07:06.590605 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-5521d2f0-d833-4707-9d3d-7e9fd1357fe5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.582575 Jul 19 11:07:06.591314 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 217/417] 10.222.0.33 () {60 vars in 1146 bytes} [Tue Jul 19 11:07:06 2022] POST /compute/v2.1/servers => generated 401 bytes in 584 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 19 11:07:06.600885 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-177868ac-91dc-45b1-91bb-dfc047f1f1f1 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:06.601500 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-177868ac-91dc-45b1-91bb-dfc047f1f1f1 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 988f14d7-6354-4388-a51c-b4df9749c671] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:06.644043 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b2075aa5-97ee-448e-aa18-ab4f64ca55b7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:06.652409 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-b2075aa5-97ee-448e-aa18-ab4f64ca55b7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1503 microversion: 2.1 time: 0.155910 Jul 19 11:07:06.663157 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 201/418] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:06 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1503 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:07:06.663452 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-177868ac-91dc-45b1-91bb-dfc047f1f1f1 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:06.663452 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-177868ac-91dc-45b1-91bb-dfc047f1f1f1 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671" status: 200 len: 1239 microversion: 2.1 time: 0.065973 Jul 19 11:07:06.663993 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 218/419] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:06 2022] GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671 => generated 1239 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:07:06.669834 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-434139a5-a9f2-4eb9-82bf-9990db386847 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "26e5a71b-5371-4421-9947-030ea3f505fe"}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:07:06.670955 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-434139a5-a9f2-4eb9-82bf-9990db386847 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:06.677071 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-434139a5-a9f2-4eb9-82bf-9990db386847 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:06.677514 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-434139a5-a9f2-4eb9-82bf-9990db386847 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:06.678296 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-434139a5-a9f2-4eb9-82bf-9990db386847 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:06.678710 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-434139a5-a9f2-4eb9-82bf-9990db386847 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:06.701410 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.wsgi [None req-434139a5-a9f2-4eb9-82bf-9990db386847 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] HTTP exception thrown: Cannot 'attach_volume' instance c3ea5a95-376a-4482-b664-f069e803e258 while it is in vm_state rescued Jul 19 11:07:06.703301 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-434139a5-a9f2-4eb9-82bf-9990db386847 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Request method failure captured: Jul 19 11:07:06.703301 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: request: POST /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258/os-volume_attachments HTTP/1.1 Jul 19 11:07:06.703301 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept: application/json Jul 19 11:07:06.703301 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept-Encoding: identity Jul 19 11:07:06.703301 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Connection: close Jul 19 11:07:06.703301 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Length: 74 Jul 19 11:07:06.703301 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Type: application/json Jul 19 11:07:06.703301 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Host: 10.222.0.33 Jul 19 11:07:06.703301 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Openstack-System-Scope: None Jul 19 11:07:06.703301 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: User-Agent: python-urllib3/1.26.9 Jul 19 11:07:06.703301 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Auth-Token: gAAAAABi1o_D_huvBlD9ZJOIcq8Z1aOSnILmJp8MQgaQ8MuWsEAL_ZmZPsH--WyF20ahgvN08GveS7vk1bvyoVIPWYbHaxCp791BKfyC6yZ04rwPPH3vQlTPlGO54x_Ovlr-nuJTMqDQHYocrhpCc4nPGpje8DwiZIJCWR_YJdRwNKP8_4bwd5o Jul 19 11:07:06.703301 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Id: None Jul 19 11:07:06.703301 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Name: None Jul 19 11:07:06.703301 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Identity-Status: Confirmed Jul 19 11:07:06.703301 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Is-Admin-Project: True Jul 19 11:07:06.703301 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Id: default Jul 19 11:07:06.703301 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Name: Default Jul 19 11:07:06.703301 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Id: 05fceba2c70c40a399e71daabe8edef6 Jul 19 11:07:06.703301 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1654338802 Jul 19 11:07:06.703301 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Role: member,reader Jul 19 11:07:06.703301 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Roles: member,reader Jul 19 11:07:06.704577 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/05fceba2c70c40a399e71daabe8edef6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/05fceba2c70c40a399e71daabe8edef6"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/05fceba2c70c40a399e71daabe8edef6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_05fceba2c70c40a399e71daabe8edef6"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:07:06.704577 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1654338802 Jul 19 11:07:06.704577 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Id: 05fceba2c70c40a399e71daabe8edef6 Jul 19 11:07:06.704577 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1654338802 Jul 19 11:07:06.704577 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User: tempest-ServerRescueNegativeTestJSON-1654338802-project Jul 19 11:07:06.704577 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Id: default Jul 19 11:07:06.704577 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Name: Default Jul 19 11:07:06.704577 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Id: c95281b28eae450d8678583a929c3b8a Jul 19 11:07:06.704577 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1654338802-project Jul 19 11:07:06.704577 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Jul 19 11:07:06.704577 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {"volumeAttachment": {"volumeId": "26e5a71b-5371-4421-9947-030ea3f505fe"}} Jul 19 11:07:06.704577 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: method: > Jul 19 11:07:06.705396 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: exception: Cannot 'attach_volume' instance c3ea5a95-376a-4482-b664-f069e803e258 while it is in vm_state rescued Jul 19 11:07:06.705396 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: action_args: {'server_id': 'c3ea5a95-376a-4482-b664-f069e803e258', 'body': {'volumeAttachment': {'volumeId': '26e5a71b-5371-4421-9947-030ea3f505fe'}}} Jul 19 11:07:06.705396 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:07:06.705396 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:07:06.705396 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Jul 19 11:07:06.705396 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Jul 19 11:07:06.705396 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 19 11:07:06.705396 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 19 11:07:06.705396 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Jul 19 11:07:06.705396 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Jul 19 11:07:06.705396 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance c3ea5a95-376a-4482-b664-f069e803e258 in vm_state rescued. Cannot attach_volume while the instance is in this state. Jul 19 11:07:06.705396 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:07:06.705396 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:07:06.705396 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:07:06.705396 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:07:06.705396 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:07:06.705396 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:07:06.705396 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:07:06.705396 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:07:06.706522 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:07:06.706522 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:07:06.706522 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 19 11:07:06.706522 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 19 11:07:06.706522 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 19 11:07:06.706522 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 19 11:07:06.706522 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 19 11:07:06.706522 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 19 11:07:06.706522 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Jul 19 11:07:06.706522 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Jul 19 11:07:06.706522 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Jul 19 11:07:06.706522 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Jul 19 11:07:06.706522 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance c3ea5a95-376a-4482-b664-f069e803e258 while it is in vm_state rescued Jul 19 11:07:06.706522 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:07:06.706522 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:07:06.706522 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:07:06.706522 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:07:06.706522 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:07:06.706522 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:07:06.707624 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:07:06.707624 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:07:06.707624 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance c3ea5a95-376a-4482-b664-f069e803e258 while it is in vm_state rescued Jul 19 11:07:06.707624 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:07:06.707624 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-434139a5-a9f2-4eb9-82bf-9990db386847 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Returning 409 to user: Cannot 'attach_volume' instance c3ea5a95-376a-4482-b664-f069e803e258 while it is in vm_state rescued {{(pid=117849) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:07:06.707624 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-434139a5-a9f2-4eb9-82bf-9990db386847 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "POST /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.039011 Jul 19 11:07:06.707624 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 202/420] 10.222.0.33 () {60 vars in 1322 bytes} [Tue Jul 19 11:07:06 2022] POST /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258/os-volume_attachments => generated 152 bytes in 41 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Jul 19 11:07:06.714345 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-75f8e3f7-1adb-434e-8d26-3006d34e7de6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:07:06.714827 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-75f8e3f7-1adb-434e-8d26-3006d34e7de6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:06.722263 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-75f8e3f7-1adb-434e-8d26-3006d34e7de6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:06.722806 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-75f8e3f7-1adb-434e-8d26-3006d34e7de6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:06.723471 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-75f8e3f7-1adb-434e-8d26-3006d34e7de6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:06.723907 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-75f8e3f7-1adb-434e-8d26-3006d34e7de6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:06.742919 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=100 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:07:06.797358 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-75f8e3f7-1adb-434e-8d26-3006d34e7de6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "POST /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258/action" status: 202 len: 0 microversion: 2.1 time: 0.087143 Jul 19 11:07:06.798391 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 219/421] 10.222.0.33 () {60 vars in 1277 bytes} [Tue Jul 19 11:07:06 2022] POST /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258/action => generated 0 bytes in 89 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 19 11:07:06.806481 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-960a98d1-cb48-4406-8495-907ceece1b91 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:06.807086 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-960a98d1-cb48-4406-8495-907ceece1b91 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:06.814656 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-960a98d1-cb48-4406-8495-907ceece1b91 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:06.815453 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-960a98d1-cb48-4406-8495-907ceece1b91 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:06.816552 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-960a98d1-cb48-4406-8495-907ceece1b91 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:06.817360 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-960a98d1-cb48-4406-8495-907ceece1b91 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:06.821647 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-61785272-a864-4b23-89b8-f9eb42e59c2a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:06.823013 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-61785272-a864-4b23-89b8-f9eb42e59c2a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:06.833063 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-61785272-a864-4b23-89b8-f9eb42e59c2a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:06.833728 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-61785272-a864-4b23-89b8-f9eb42e59c2a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:06.834631 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-61785272-a864-4b23-89b8-f9eb42e59c2a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:06.835257 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-61785272-a864-4b23-89b8-f9eb42e59c2a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:07.069202 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-61785272-a864-4b23-89b8-f9eb42e59c2a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:07.082215 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-61785272-a864-4b23-89b8-f9eb42e59c2a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1259 microversion: 2.1 time: 0.265809 Jul 19 11:07:07.084351 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 220/422] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:06 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1259 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:07.190140 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-960a98d1-cb48-4406-8495-907ceece1b91 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:07.199487 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-960a98d1-cb48-4406-8495-907ceece1b91 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1511 microversion: 2.1 time: 0.396937 Jul 19 11:07:07.200059 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 203/423] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:06 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1511 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:07:07.318017 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-1898c001-b54b-4db3-8688-836bd5e135f8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:07.318729 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-1898c001-b54b-4db3-8688-836bd5e135f8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:07.331330 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1898c001-b54b-4db3-8688-836bd5e135f8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:07.332187 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1898c001-b54b-4db3-8688-836bd5e135f8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:07.333203 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1898c001-b54b-4db3-8688-836bd5e135f8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:07.333914 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1898c001-b54b-4db3-8688-836bd5e135f8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:07.568877 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-1898c001-b54b-4db3-8688-836bd5e135f8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:07.583554 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-1898c001-b54b-4db3-8688-836bd5e135f8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:07.590593 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-1898c001-b54b-4db3-8688-836bd5e135f8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:07.592992 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-1898c001-b54b-4db3-8688-836bd5e135f8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1722 microversion: 2.20 time: 0.280001 Jul 19 11:07:07.594130 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 221/424] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:07 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1722 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:07.682137 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-79d81e00-b7fd-4570-8422-15acf5d0932d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:07.682866 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-79d81e00-b7fd-4570-8422-15acf5d0932d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 988f14d7-6354-4388-a51c-b4df9749c671] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:07.691290 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-79d81e00-b7fd-4570-8422-15acf5d0932d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:07.691879 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-79d81e00-b7fd-4570-8422-15acf5d0932d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:07.692849 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-79d81e00-b7fd-4570-8422-15acf5d0932d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:07.693351 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-79d81e00-b7fd-4570-8422-15acf5d0932d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:07.764891 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-79d81e00-b7fd-4570-8422-15acf5d0932d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:07.773244 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-79d81e00-b7fd-4570-8422-15acf5d0932d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671" status: 200 len: 1291 microversion: 2.1 time: 0.095844 Jul 19 11:07:07.774031 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 204/425] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:07 2022] GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671 => generated 1291 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:08.103620 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-56b58ec7-afe1-4ff3-a7d6-7c56b214ed51 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:08.104669 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-56b58ec7-afe1-4ff3-a7d6-7c56b214ed51 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:08.119566 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-56b58ec7-afe1-4ff3-a7d6-7c56b214ed51 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:08.120473 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-56b58ec7-afe1-4ff3-a7d6-7c56b214ed51 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:08.121392 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-56b58ec7-afe1-4ff3-a7d6-7c56b214ed51 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:08.122986 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-56b58ec7-afe1-4ff3-a7d6-7c56b214ed51 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:08.217336 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-61eceda9-9d02-4aab-80ce-e7a8c319baf8 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:08.218551 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-61eceda9-9d02-4aab-80ce-e7a8c319baf8 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:08.229504 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-61eceda9-9d02-4aab-80ce-e7a8c319baf8 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:08.230380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-61eceda9-9d02-4aab-80ce-e7a8c319baf8 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:08.231351 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-61eceda9-9d02-4aab-80ce-e7a8c319baf8 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:08.231784 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-61eceda9-9d02-4aab-80ce-e7a8c319baf8 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:08.305943 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-56b58ec7-afe1-4ff3-a7d6-7c56b214ed51 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:08.314712 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-56b58ec7-afe1-4ff3-a7d6-7c56b214ed51 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1259 microversion: 2.1 time: 0.218625 Jul 19 11:07:08.315283 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 222/426] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:08 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1259 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:08.389497 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-61eceda9-9d02-4aab-80ce-e7a8c319baf8 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:08.397202 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-61eceda9-9d02-4aab-80ce-e7a8c319baf8 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1511 microversion: 2.1 time: 0.183648 Jul 19 11:07:08.397763 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 205/427] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:08 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1511 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:07:08.613987 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-41b4b784-708a-4b7b-9d8b-60b8c8cd6daa tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:08.614592 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-41b4b784-708a-4b7b-9d8b-60b8c8cd6daa tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:08.621001 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-41b4b784-708a-4b7b-9d8b-60b8c8cd6daa tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:08.621447 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-41b4b784-708a-4b7b-9d8b-60b8c8cd6daa tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:08.622241 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-41b4b784-708a-4b7b-9d8b-60b8c8cd6daa tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:08.622916 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-41b4b784-708a-4b7b-9d8b-60b8c8cd6daa tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:08.762270 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-41b4b784-708a-4b7b-9d8b-60b8c8cd6daa tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:08.770335 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-41b4b784-708a-4b7b-9d8b-60b8c8cd6daa tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:08.771857 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-41b4b784-708a-4b7b-9d8b-60b8c8cd6daa tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:08.773027 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-41b4b784-708a-4b7b-9d8b-60b8c8cd6daa tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1722 microversion: 2.20 time: 0.162935 Jul 19 11:07:08.774172 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 223/428] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:08 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1722 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:08.792397 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-4d23c7c0-5fc7-4a37-a603-b47bc616d51b tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:08.793043 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-4d23c7c0-5fc7-4a37-a603-b47bc616d51b tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 988f14d7-6354-4388-a51c-b4df9749c671] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:08.799072 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4d23c7c0-5fc7-4a37-a603-b47bc616d51b tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:08.799510 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4d23c7c0-5fc7-4a37-a603-b47bc616d51b tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:08.800261 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4d23c7c0-5fc7-4a37-a603-b47bc616d51b tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:08.800688 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4d23c7c0-5fc7-4a37-a603-b47bc616d51b tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:08.875356 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-4d23c7c0-5fc7-4a37-a603-b47bc616d51b tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:08.882666 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-4d23c7c0-5fc7-4a37-a603-b47bc616d51b tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671" status: 200 len: 1309 microversion: 2.1 time: 0.094044 Jul 19 11:07:08.883714 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 206/429] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:08 2022] GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671 => generated 1309 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:09.332284 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-794597ed-c018-415f-a850-31bfc6e01fb2 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:09.333659 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-794597ed-c018-415f-a850-31bfc6e01fb2 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:09.340426 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-794597ed-c018-415f-a850-31bfc6e01fb2 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:09.341001 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-794597ed-c018-415f-a850-31bfc6e01fb2 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:09.341859 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-794597ed-c018-415f-a850-31bfc6e01fb2 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:09.342566 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-794597ed-c018-415f-a850-31bfc6e01fb2 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:09.415301 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-2eb099de-447e-4815-8dce-f7b9fc3ee5f9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:09.416149 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-2eb099de-447e-4815-8dce-f7b9fc3ee5f9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:09.423025 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2eb099de-447e-4815-8dce-f7b9fc3ee5f9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:09.423469 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2eb099de-447e-4815-8dce-f7b9fc3ee5f9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:09.424232 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2eb099de-447e-4815-8dce-f7b9fc3ee5f9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:09.424679 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2eb099de-447e-4815-8dce-f7b9fc3ee5f9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:09.576034 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-794597ed-c018-415f-a850-31bfc6e01fb2 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:09.581579 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-2eb099de-447e-4815-8dce-f7b9fc3ee5f9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:09.589889 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-2eb099de-447e-4815-8dce-f7b9fc3ee5f9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1511 microversion: 2.1 time: 0.179037 Jul 19 11:07:09.590069 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-794597ed-c018-415f-a850-31bfc6e01fb2 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1259 microversion: 2.1 time: 0.262030 Jul 19 11:07:09.590477 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 224/430] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:09 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1259 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:09.592783 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 207/431] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:09 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1511 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:07:09.793736 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-a8d8bf89-41ed-4e72-bca2-0345caa35c0f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:09.795289 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-a8d8bf89-41ed-4e72-bca2-0345caa35c0f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:09.803698 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a8d8bf89-41ed-4e72-bca2-0345caa35c0f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:09.804249 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a8d8bf89-41ed-4e72-bca2-0345caa35c0f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:09.805418 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a8d8bf89-41ed-4e72-bca2-0345caa35c0f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:09.805943 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a8d8bf89-41ed-4e72-bca2-0345caa35c0f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:09.901206 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-f9ba12b8-32bb-47b8-9556-243dc1793cc5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:09.901799 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-f9ba12b8-32bb-47b8-9556-243dc1793cc5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 988f14d7-6354-4388-a51c-b4df9749c671] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:09.907762 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f9ba12b8-32bb-47b8-9556-243dc1793cc5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:09.908215 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f9ba12b8-32bb-47b8-9556-243dc1793cc5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:09.909002 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f9ba12b8-32bb-47b8-9556-243dc1793cc5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:09.909434 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f9ba12b8-32bb-47b8-9556-243dc1793cc5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:09.976016 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-f9ba12b8-32bb-47b8-9556-243dc1793cc5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:09.984547 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-f9ba12b8-32bb-47b8-9556-243dc1793cc5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671" status: 200 len: 1297 microversion: 2.1 time: 0.087692 Jul 19 11:07:09.985218 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 208/432] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:09 2022] GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671 => generated 1297 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:10.004261 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-a8d8bf89-41ed-4e72-bca2-0345caa35c0f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:10.014398 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-a8d8bf89-41ed-4e72-bca2-0345caa35c0f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:10.015983 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-a8d8bf89-41ed-4e72-bca2-0345caa35c0f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:10.017172 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-a8d8bf89-41ed-4e72-bca2-0345caa35c0f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1722 microversion: 2.20 time: 0.228997 Jul 19 11:07:10.017728 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 225/433] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:09 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1722 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:10.222998 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-4101a554-3aa7-4c5d-b6cb-7c1b36379799 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-840871984", "imageRef": "3085883e-2289-4314-aa47-9c41d4368b02", "flavorRef": "452"}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:07:10.276058 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.network.neutron [None req-4101a554-3aa7-4c5d-b6cb-7c1b36379799 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] validate_networks() for None {{(pid=117849) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 19 11:07:10.607221 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-d9f1802a-8875-4ad1-9334-3267eb6c4c81 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:10.607965 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-d9f1802a-8875-4ad1-9334-3267eb6c4c81 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:10.620072 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d9f1802a-8875-4ad1-9334-3267eb6c4c81 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:10.621108 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d9f1802a-8875-4ad1-9334-3267eb6c4c81 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:10.621892 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d9f1802a-8875-4ad1-9334-3267eb6c4c81 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:10.622353 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d9f1802a-8875-4ad1-9334-3267eb6c4c81 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:10.650024 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-4101a554-3aa7-4c5d-b6cb-7c1b36379799 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Flavor limits 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 19 11:07:10.650530 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-4101a554-3aa7-4c5d-b6cb-7c1b36379799 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Image limits 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 19 11:07:10.651457 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-4101a554-3aa7-4c5d-b6cb-7c1b36379799 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Flavor pref 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 19 11:07:10.652043 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-4101a554-3aa7-4c5d-b6cb-7c1b36379799 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Image pref 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 19 11:07:10.652615 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-4101a554-3aa7-4c5d-b6cb-7c1b36379799 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 19 11:07:10.672158 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4101a554-3aa7-4c5d-b6cb-7c1b36379799 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:10.673956 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4101a554-3aa7-4c5d-b6cb-7c1b36379799 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:10.674940 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4101a554-3aa7-4c5d-b6cb-7c1b36379799 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:10.676133 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4101a554-3aa7-4c5d-b6cb-7c1b36379799 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:10.714478 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-4101a554-3aa7-4c5d-b6cb-7c1b36379799 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Getting quotas for project 2b4575dc494d40fda826dbcd0600a76a. Resources: {'cores', 'instances', 'ram'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:07:10.726215 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-4101a554-3aa7-4c5d-b6cb-7c1b36379799 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Getting quotas for user 49e762b40bfd49859d5820d3e45adb1b and project 2b4575dc494d40fda826dbcd0600a76a. Resources: {'cores', 'instances', 'ram'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:07:10.755722 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-4101a554-3aa7-4c5d-b6cb-7c1b36379799 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Going to run 1 instances... {{(pid=117849) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 19 11:07:10.760112 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-4101a554-3aa7-4c5d-b6cb-7c1b36379799 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] [instance: 1c66adb2-72ed-4327-ba4c-efdef9fa18f2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='3085883e-2289-4314-aa47-9c41d4368b02',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117849) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 19 11:07:10.816975 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-d9f1802a-8875-4ad1-9334-3267eb6c4c81 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:10.830154 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-4101a554-3aa7-4c5d-b6cb-7c1b36379799 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.610878 Jul 19 11:07:10.831826 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 209/434] 10.222.0.33 () {60 vars in 1146 bytes} [Tue Jul 19 11:07:10 2022] POST /compute/v2.1/servers => generated 380 bytes in 613 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 19 11:07:10.837732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-c640351b-3985-45a0-8189-e14c22a20db7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:10.838379 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-c640351b-3985-45a0-8189-e14c22a20db7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:10.845279 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c640351b-3985-45a0-8189-e14c22a20db7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:10.845755 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c640351b-3985-45a0-8189-e14c22a20db7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:10.846507 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c640351b-3985-45a0-8189-e14c22a20db7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:10.846975 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c640351b-3985-45a0-8189-e14c22a20db7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:10.849023 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-d9f1802a-8875-4ad1-9334-3267eb6c4c81 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1259 microversion: 2.1 time: 0.245501 Jul 19 11:07:10.862390 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 226/435] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:10 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1259 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:07:10.872657 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-510bc04d-7eac-4d39-b51b-dfe0f6d929cf tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:10.873770 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-510bc04d-7eac-4d39-b51b-dfe0f6d929cf tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] [instance: 1c66adb2-72ed-4327-ba4c-efdef9fa18f2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:10.939226 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-510bc04d-7eac-4d39-b51b-dfe0f6d929cf tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '49e762b40bfd49859d5820d3e45adb1b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b4575dc494d40fda826dbcd0600a76a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:10.947890 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-510bc04d-7eac-4d39-b51b-dfe0f6d929cf tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] 10.222.0.33 "GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2" status: 200 len: 1214 microversion: 2.1 time: 0.084144 Jul 19 11:07:10.948495 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 227/436] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:10 2022] GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2 => generated 1214 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:11.006016 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-b0078f22-2f13-4f72-8669-2aae4b4c310f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:11.006016 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-b0078f22-2f13-4f72-8669-2aae4b4c310f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 988f14d7-6354-4388-a51c-b4df9749c671] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:11.012046 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b0078f22-2f13-4f72-8669-2aae4b4c310f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:11.012494 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b0078f22-2f13-4f72-8669-2aae4b4c310f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:11.013519 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b0078f22-2f13-4f72-8669-2aae4b4c310f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:11.014154 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b0078f22-2f13-4f72-8669-2aae4b4c310f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:11.129210 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-b0078f22-2f13-4f72-8669-2aae4b4c310f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:11.139328 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-b0078f22-2f13-4f72-8669-2aae4b4c310f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671" status: 200 len: 1297 microversion: 2.1 time: 0.138907 Jul 19 11:07:11.140340 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 228/437] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:10 2022] GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671 => generated 1297 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:07:11.147202 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-2ac6df7e-4518-4a78-8206-3c17772efcad tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:11.148323 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-2ac6df7e-4518-4a78-8206-3c17772efcad tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:11.168341 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2ac6df7e-4518-4a78-8206-3c17772efcad tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:11.169903 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2ac6df7e-4518-4a78-8206-3c17772efcad tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:11.170879 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2ac6df7e-4518-4a78-8206-3c17772efcad tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:11.171451 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2ac6df7e-4518-4a78-8206-3c17772efcad tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:11.315858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-c640351b-3985-45a0-8189-e14c22a20db7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:11.323959 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-c640351b-3985-45a0-8189-e14c22a20db7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1517 microversion: 2.1 time: 0.490918 Jul 19 11:07:11.324534 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 210/438] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:10 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1517 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:11.497425 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-2ac6df7e-4518-4a78-8206-3c17772efcad tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:11.517642 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-2ac6df7e-4518-4a78-8206-3c17772efcad tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:11.520725 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-2ac6df7e-4518-4a78-8206-3c17772efcad tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:11.522516 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-2ac6df7e-4518-4a78-8206-3c17772efcad tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1722 microversion: 2.20 time: 0.380678 Jul 19 11:07:11.531306 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 229/439] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:11 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1722 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:11.874335 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-0bfbd307-86f5-4c97-9994-a3c916618c08 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:11.875431 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-0bfbd307-86f5-4c97-9994-a3c916618c08 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:11.886953 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0bfbd307-86f5-4c97-9994-a3c916618c08 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:11.887731 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0bfbd307-86f5-4c97-9994-a3c916618c08 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:11.889222 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0bfbd307-86f5-4c97-9994-a3c916618c08 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:11.890005 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0bfbd307-86f5-4c97-9994-a3c916618c08 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:11.971135 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-0d18820a-9fa9-4556-86e4-865ec83dcb7d tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:11.972135 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-0d18820a-9fa9-4556-86e4-865ec83dcb7d tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] [instance: 1c66adb2-72ed-4327-ba4c-efdef9fa18f2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:11.991597 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0d18820a-9fa9-4556-86e4-865ec83dcb7d tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:11.992045 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0d18820a-9fa9-4556-86e4-865ec83dcb7d tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:12.009759 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0d18820a-9fa9-4556-86e4-865ec83dcb7d tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:12.010217 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0d18820a-9fa9-4556-86e4-865ec83dcb7d tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:12.081578 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-0bfbd307-86f5-4c97-9994-a3c916618c08 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:12.091846 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-0bfbd307-86f5-4c97-9994-a3c916618c08 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1259 microversion: 2.1 time: 0.224803 Jul 19 11:07:12.093152 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 211/440] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:11 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1259 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:12.112817 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-0d18820a-9fa9-4556-86e4-865ec83dcb7d tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '49e762b40bfd49859d5820d3e45adb1b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b4575dc494d40fda826dbcd0600a76a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:12.124094 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-0d18820a-9fa9-4556-86e4-865ec83dcb7d tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] 10.222.0.33 "GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2" status: 200 len: 1218 microversion: 2.1 time: 0.157725 Jul 19 11:07:12.125237 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 230/441] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:11 2022] GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2 => generated 1218 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:12.165025 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-9aee76de-6be5-42b6-8b68-3bc77f4a5ea8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:12.165647 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-9aee76de-6be5-42b6-8b68-3bc77f4a5ea8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 988f14d7-6354-4388-a51c-b4df9749c671] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:12.177485 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9aee76de-6be5-42b6-8b68-3bc77f4a5ea8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:12.178087 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9aee76de-6be5-42b6-8b68-3bc77f4a5ea8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:12.179101 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9aee76de-6be5-42b6-8b68-3bc77f4a5ea8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:12.180490 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9aee76de-6be5-42b6-8b68-3bc77f4a5ea8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:12.390357 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-9aee76de-6be5-42b6-8b68-3bc77f4a5ea8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:12.398230 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-9aee76de-6be5-42b6-8b68-3bc77f4a5ea8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671" status: 200 len: 1364 microversion: 2.1 time: 0.243366 Jul 19 11:07:12.409386 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 212/442] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:12 2022] GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671 => generated 1364 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:12.550430 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-082a9fa0-bc73-4f1f-89d8-b5b9b8f4be63 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:12.551346 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-082a9fa0-bc73-4f1f-89d8-b5b9b8f4be63 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:12.565754 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-082a9fa0-bc73-4f1f-89d8-b5b9b8f4be63 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:12.566329 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-082a9fa0-bc73-4f1f-89d8-b5b9b8f4be63 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:12.567551 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-082a9fa0-bc73-4f1f-89d8-b5b9b8f4be63 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:12.568384 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-082a9fa0-bc73-4f1f-89d8-b5b9b8f4be63 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:12.717953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-082a9fa0-bc73-4f1f-89d8-b5b9b8f4be63 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:12.728422 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-082a9fa0-bc73-4f1f-89d8-b5b9b8f4be63 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:12.730147 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-082a9fa0-bc73-4f1f-89d8-b5b9b8f4be63 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:12.731930 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-082a9fa0-bc73-4f1f-89d8-b5b9b8f4be63 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1722 microversion: 2.20 time: 0.186039 Jul 19 11:07:12.733123 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 231/443] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:12 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1722 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:12.864836 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-a78af58b-0ce7-444c-857a-a0d44182802c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Action: 'create', calling method: .version_select at 0x7f04aa5ae310>, body: {"keypair": {"name": "tempest-keypair-1183558961", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBG0E0DP2ErTMQKTj49J4q/70F07qZb3yRbrfEqI1bqaJNsxfMwAsQZr+y3CnZNKJyVzGcxciUDPI5Ero5CQ/Ji5frIwSF+dDd63tuc0YgrqIjZSW3eGnt0q/02BODTrPAg=="}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:07:12.884340 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-a78af58b-0ce7-444c-857a-a0d44182802c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Getting quotas for project 05fceba2c70c40a399e71daabe8edef6. Resources: {'key_pairs'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:07:12.891237 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-a78af58b-0ce7-444c-857a-a0d44182802c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Getting quotas for user c95281b28eae450d8678583a929c3b8a and project 05fceba2c70c40a399e71daabe8edef6. Resources: {'key_pairs'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:07:12.953826 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-a78af58b-0ce7-444c-857a-a0d44182802c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.092859 Jul 19 11:07:12.955195 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 213/444] 10.222.0.33 () {60 vars in 1158 bytes} [Tue Jul 19 11:07:12 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:07:12.957938 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-a9d64140-226e-40fe-8505-e2ce4a61ab74 req-3886743d-2722-49fe-b8c3-99959c106379 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "988f14d7-6354-4388-a51c-b4df9749c671", "tag": "ebf84be3-6be9-46a9-b6fd-10aca9db5a6f"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:07:12.981238 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-a9d64140-226e-40fe-8505-e2ce4a61ab74 req-3886743d-2722-49fe-b8c3-99959c106379 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:12.981806 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-a9d64140-226e-40fe-8505-e2ce4a61ab74 req-3886743d-2722-49fe-b8c3-99959c106379 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:13.005245 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-a9d64140-226e-40fe-8505-e2ce4a61ab74 req-3886743d-2722-49fe-b8c3-99959c106379 service nova] Creating event network-changed:ebf84be3-6be9-46a9-b6fd-10aca9db5a6f for instance 988f14d7-6354-4388-a51c-b4df9749c671 on hv-cinder-maste Jul 19 11:07:13.011667 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-a9d64140-226e-40fe-8505-e2ce4a61ab74 req-3886743d-2722-49fe-b8c3-99959c106379 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.057387 Jul 19 11:07:13.012279 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 232/445] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:07:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:07:13.109962 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-a003620a-537e-489f-b6c6-e20e846e429b tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:13.110577 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-a003620a-537e-489f-b6c6-e20e846e429b tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:13.118907 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a003620a-537e-489f-b6c6-e20e846e429b tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:13.119357 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a003620a-537e-489f-b6c6-e20e846e429b tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:13.120319 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a003620a-537e-489f-b6c6-e20e846e429b tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:13.122075 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a003620a-537e-489f-b6c6-e20e846e429b tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:13.149334 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-b9ad41cc-7415-4c70-8e25-3396b353ac7d tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:13.151127 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-b9ad41cc-7415-4c70-8e25-3396b353ac7d tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] [instance: 1c66adb2-72ed-4327-ba4c-efdef9fa18f2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:13.154707 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=4 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:07:13.162399 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b9ad41cc-7415-4c70-8e25-3396b353ac7d tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:13.163295 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b9ad41cc-7415-4c70-8e25-3396b353ac7d tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:13.164804 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b9ad41cc-7415-4c70-8e25-3396b353ac7d tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:13.165469 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b9ad41cc-7415-4c70-8e25-3396b353ac7d tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:13.332585 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-b9ad41cc-7415-4c70-8e25-3396b353ac7d tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '49e762b40bfd49859d5820d3e45adb1b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b4575dc494d40fda826dbcd0600a76a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:13.354562 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-b9ad41cc-7415-4c70-8e25-3396b353ac7d tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] 10.222.0.33 "GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2" status: 200 len: 1274 microversion: 2.1 time: 0.211954 Jul 19 11:07:13.368690 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 233/446] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:13 2022] GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2 => generated 1274 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:13.415880 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-5fbb8cf9-1a61-4a2a-bb1a-dbe4479fbcda tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:13.416669 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-5fbb8cf9-1a61-4a2a-bb1a-dbe4479fbcda tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 988f14d7-6354-4388-a51c-b4df9749c671] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:13.425824 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5fbb8cf9-1a61-4a2a-bb1a-dbe4479fbcda tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:13.429341 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5fbb8cf9-1a61-4a2a-bb1a-dbe4479fbcda tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:13.430476 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5fbb8cf9-1a61-4a2a-bb1a-dbe4479fbcda tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:13.431120 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5fbb8cf9-1a61-4a2a-bb1a-dbe4479fbcda tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:13.664983 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-a003620a-537e-489f-b6c6-e20e846e429b tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:13.667933 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-5fbb8cf9-1a61-4a2a-bb1a-dbe4479fbcda tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:13.674828 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-a003620a-537e-489f-b6c6-e20e846e429b tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1259 microversion: 2.1 time: 0.569180 Jul 19 11:07:13.675551 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 214/447] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:13 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1259 bytes in 571 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:13.676755 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-5fbb8cf9-1a61-4a2a-bb1a-dbe4479fbcda tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671" status: 200 len: 1364 microversion: 2.1 time: 0.265171 Jul 19 11:07:13.688832 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 234/448] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:13 2022] GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671 => generated 1364 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:13.750918 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-735c8436-bc56-4f76-bffa-5e11c59c31e8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:13.751676 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-735c8436-bc56-4f76-bffa-5e11c59c31e8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:13.758700 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-735c8436-bc56-4f76-bffa-5e11c59c31e8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:13.759441 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-735c8436-bc56-4f76-bffa-5e11c59c31e8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:13.760431 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-735c8436-bc56-4f76-bffa-5e11c59c31e8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:13.761473 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-735c8436-bc56-4f76-bffa-5e11c59c31e8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:14.370714 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-735c8436-bc56-4f76-bffa-5e11c59c31e8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:14.375537 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-e2efb2fd-955d-4295-953f-29febfc2d6f2 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:14.376218 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-e2efb2fd-955d-4295-953f-29febfc2d6f2 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] [instance: 1c66adb2-72ed-4327-ba4c-efdef9fa18f2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:14.385310 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-735c8436-bc56-4f76-bffa-5e11c59c31e8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:14.386886 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e2efb2fd-955d-4295-953f-29febfc2d6f2 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:14.387354 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e2efb2fd-955d-4295-953f-29febfc2d6f2 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:14.387678 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-735c8436-bc56-4f76-bffa-5e11c59c31e8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:14.388189 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e2efb2fd-955d-4295-953f-29febfc2d6f2 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:14.389481 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-735c8436-bc56-4f76-bffa-5e11c59c31e8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1722 microversion: 2.20 time: 0.642778 Jul 19 11:07:14.389673 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e2efb2fd-955d-4295-953f-29febfc2d6f2 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:14.390470 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 215/449] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:13 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1722 bytes in 645 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:14.470609 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-e2efb2fd-955d-4295-953f-29febfc2d6f2 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '49e762b40bfd49859d5820d3e45adb1b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b4575dc494d40fda826dbcd0600a76a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:14.486206 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-e2efb2fd-955d-4295-953f-29febfc2d6f2 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] 10.222.0.33 "GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2" status: 200 len: 1272 microversion: 2.1 time: 0.115755 Jul 19 11:07:14.498718 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 235/450] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:14 2022] GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2 => generated 1272 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:07:14.691570 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-658ce48b-0521-47df-b6d3-9eb12556d2c3 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:14.692182 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-658ce48b-0521-47df-b6d3-9eb12556d2c3 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:14.700202 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-658ce48b-0521-47df-b6d3-9eb12556d2c3 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:14.701565 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-0c4e3702-381c-4ac6-a417-2520b4495728 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:14.703758 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-0c4e3702-381c-4ac6-a417-2520b4495728 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 988f14d7-6354-4388-a51c-b4df9749c671] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:14.705975 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-658ce48b-0521-47df-b6d3-9eb12556d2c3 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:14.706859 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-658ce48b-0521-47df-b6d3-9eb12556d2c3 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:14.707477 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-658ce48b-0521-47df-b6d3-9eb12556d2c3 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:14.747975 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0c4e3702-381c-4ac6-a417-2520b4495728 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:14.753482 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0c4e3702-381c-4ac6-a417-2520b4495728 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:14.758144 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0c4e3702-381c-4ac6-a417-2520b4495728 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:14.758871 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0c4e3702-381c-4ac6-a417-2520b4495728 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:15.031509 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-0c4e3702-381c-4ac6-a417-2520b4495728 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:15.034900 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-658ce48b-0521-47df-b6d3-9eb12556d2c3 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:15.035557 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:07:15.041917 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-0c4e3702-381c-4ac6-a417-2520b4495728 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671" status: 200 len: 1364 microversion: 2.1 time: 0.346947 Jul 19 11:07:15.052424 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 236/451] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:14 2022] GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671 => generated 1364 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:15.054768 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-658ce48b-0521-47df-b6d3-9eb12556d2c3 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1259 microversion: 2.1 time: 0.367072 Jul 19 11:07:15.056895 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 216/452] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:14 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1259 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:15.068781 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-c2100a44-34ac-4ecf-ac9d-d16bf7a9279d req-1a8e7683-01d4-4a38-ab17-e0e4116d08d0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5b72a14b-bd18-4dfa-a68f-12a5424f471c", "name": "network-vif-plugged", "status": "completed", "tag": "da843419-4401-4a26-82c2-7930b51aa1b8"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:07:15.078850 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-c2100a44-34ac-4ecf-ac9d-d16bf7a9279d req-1a8e7683-01d4-4a38-ab17-e0e4116d08d0 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:15.079446 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-c2100a44-34ac-4ecf-ac9d-d16bf7a9279d req-1a8e7683-01d4-4a38-ab17-e0e4116d08d0 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:15.092125 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-c2100a44-34ac-4ecf-ac9d-d16bf7a9279d req-1a8e7683-01d4-4a38-ab17-e0e4116d08d0 service nova] Creating event network-vif-plugged:da843419-4401-4a26-82c2-7930b51aa1b8 for instance 5b72a14b-bd18-4dfa-a68f-12a5424f471c on hv-cinder-maste Jul 19 11:07:15.097572 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-c2100a44-34ac-4ecf-ac9d-d16bf7a9279d req-1a8e7683-01d4-4a38-ab17-e0e4116d08d0 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043791 Jul 19 11:07:15.098290 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 237/453] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:07:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:07:15.407610 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-b82eedd5-52ab-4e52-a060-bb4c3eae648c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:15.408270 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-b82eedd5-52ab-4e52-a060-bb4c3eae648c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:15.419204 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b82eedd5-52ab-4e52-a060-bb4c3eae648c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:15.419666 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b82eedd5-52ab-4e52-a060-bb4c3eae648c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:15.420623 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b82eedd5-52ab-4e52-a060-bb4c3eae648c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:15.422736 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b82eedd5-52ab-4e52-a060-bb4c3eae648c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:15.508382 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-98542a44-bbe0-446a-99fe-314efd7be46c tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:15.509324 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-98542a44-bbe0-446a-99fe-314efd7be46c tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] [instance: 1c66adb2-72ed-4327-ba4c-efdef9fa18f2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:15.518133 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-98542a44-bbe0-446a-99fe-314efd7be46c tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:15.519076 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-98542a44-bbe0-446a-99fe-314efd7be46c tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:15.520252 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-98542a44-bbe0-446a-99fe-314efd7be46c tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:15.520939 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-98542a44-bbe0-446a-99fe-314efd7be46c tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:15.862046 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-98542a44-bbe0-446a-99fe-314efd7be46c tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '49e762b40bfd49859d5820d3e45adb1b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b4575dc494d40fda826dbcd0600a76a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:15.873485 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-98542a44-bbe0-446a-99fe-314efd7be46c tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] 10.222.0.33 "GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2" status: 200 len: 1314 microversion: 2.1 time: 0.372459 Jul 19 11:07:15.875268 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 238/454] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:15 2022] GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2 => generated 1314 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:07:15.882390 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-a1cba085-e60d-4edf-8436-f2cd77f936c0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-222978770", "imageRef": "3085883e-2289-4314-aa47-9c41d4368b02", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "1ffa3686-4e5a-4747-8994-8cc33db2a036"}], "security_groups": [{"name": "tempest-securitygroup--660548509"}], "key_name": "tempest-keypair-1183558961", "user_data": "}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:07:15.914883 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b82eedd5-52ab-4e52-a060-bb4c3eae648c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:15.950575 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b82eedd5-52ab-4e52-a060-bb4c3eae648c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:15.952312 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b82eedd5-52ab-4e52-a060-bb4c3eae648c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:15.954180 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-b82eedd5-52ab-4e52-a060-bb4c3eae648c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1722 microversion: 2.20 time: 0.550646 Jul 19 11:07:15.969162 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 217/455] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:15 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1722 bytes in 566 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:16.050098 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.network.neutron [None req-a1cba085-e60d-4edf-8436-f2cd77f936c0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] validate_networks() for [('1ffa3686-4e5a-4747-8994-8cc33db2a036', None, None, None, None, None)] {{(pid=117848) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 19 11:07:16.061735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-b3a55018-bc60-4e36-854b-7d588e092cc1 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:16.062599 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-b3a55018-bc60-4e36-854b-7d588e092cc1 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 988f14d7-6354-4388-a51c-b4df9749c671] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:16.074684 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b3a55018-bc60-4e36-854b-7d588e092cc1 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:16.075409 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b3a55018-bc60-4e36-854b-7d588e092cc1 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:16.076361 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b3a55018-bc60-4e36-854b-7d588e092cc1 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:16.076972 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b3a55018-bc60-4e36-854b-7d588e092cc1 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:16.103818 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=100 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:07:16.422803 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b3a55018-bc60-4e36-854b-7d588e092cc1 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:16.433466 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-b3a55018-bc60-4e36-854b-7d588e092cc1 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671" status: 200 len: 1364 microversion: 2.1 time: 0.376204 Jul 19 11:07:16.434717 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 218/456] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:16 2022] GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671 => generated 1364 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:16.453441 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-5960c641-b30f-4c35-a0ee-b53fa92b3c05 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:16.454826 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-5960c641-b30f-4c35-a0ee-b53fa92b3c05 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:16.464805 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5960c641-b30f-4c35-a0ee-b53fa92b3c05 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:16.466563 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5960c641-b30f-4c35-a0ee-b53fa92b3c05 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:16.468444 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5960c641-b30f-4c35-a0ee-b53fa92b3c05 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:16.469144 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5960c641-b30f-4c35-a0ee-b53fa92b3c05 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:16.518427 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell0:SELECT=2 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:07:16.604674 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-a1cba085-e60d-4edf-8436-f2cd77f936c0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Flavor limits 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 19 11:07:16.605131 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-a1cba085-e60d-4edf-8436-f2cd77f936c0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Image limits 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 19 11:07:16.605600 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-a1cba085-e60d-4edf-8436-f2cd77f936c0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Flavor pref 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 19 11:07:16.606327 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-a1cba085-e60d-4edf-8436-f2cd77f936c0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Image pref 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 19 11:07:16.606740 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-a1cba085-e60d-4edf-8436-f2cd77f936c0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 19 11:07:16.619526 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a1cba085-e60d-4edf-8436-f2cd77f936c0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:16.620181 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a1cba085-e60d-4edf-8436-f2cd77f936c0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:16.621203 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a1cba085-e60d-4edf-8436-f2cd77f936c0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:16.621889 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a1cba085-e60d-4edf-8436-f2cd77f936c0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:16.643458 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.quota [None req-a1cba085-e60d-4edf-8436-f2cd77f936c0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Getting quotas for project 05fceba2c70c40a399e71daabe8edef6. Resources: {'cores', 'instances', 'ram'} {{(pid=117848) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:07:16.650130 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.quota [None req-a1cba085-e60d-4edf-8436-f2cd77f936c0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Getting quotas for user c95281b28eae450d8678583a929c3b8a and project 05fceba2c70c40a399e71daabe8edef6. Resources: {'cores', 'instances', 'ram'} {{(pid=117848) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:07:16.664580 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-a1cba085-e60d-4edf-8436-f2cd77f936c0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Going to run 1 instances... {{(pid=117848) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 19 11:07:16.679797 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-a1cba085-e60d-4edf-8436-f2cd77f936c0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='3085883e-2289-4314-aa47-9c41d4368b02',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117848) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 19 11:07:16.764516 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-a1cba085-e60d-4edf-8436-f2cd77f936c0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 408 microversion: 2.1 time: 0.887826 Jul 19 11:07:16.769533 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 239/457] 10.222.0.33 () {60 vars in 1146 bytes} [Tue Jul 19 11:07:15 2022] POST /compute/v2.1/servers => generated 408 bytes in 893 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 19 11:07:16.786601 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-474019de-3bab-49ce-a7d6-8dc2ae76feac tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:16.788937 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-474019de-3bab-49ce-a7d6-8dc2ae76feac tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:16.897402 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-474019de-3bab-49ce-a7d6-8dc2ae76feac tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:16.905634 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-474019de-3bab-49ce-a7d6-8dc2ae76feac tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1245 microversion: 2.1 time: 0.125937 Jul 19 11:07:16.917310 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 240/458] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:16 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1245 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:16.939566 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-498dd1d9-4829-41be-9e21-4fadad62d4e0 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:16.941192 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-498dd1d9-4829-41be-9e21-4fadad62d4e0 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] [instance: 1c66adb2-72ed-4327-ba4c-efdef9fa18f2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:16.959702 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-498dd1d9-4829-41be-9e21-4fadad62d4e0 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:16.960165 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-498dd1d9-4829-41be-9e21-4fadad62d4e0 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:16.961587 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-498dd1d9-4829-41be-9e21-4fadad62d4e0 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:16.962033 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-498dd1d9-4829-41be-9e21-4fadad62d4e0 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:16.976294 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-5960c641-b30f-4c35-a0ee-b53fa92b3c05 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:16.989572 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-5960c641-b30f-4c35-a0ee-b53fa92b3c05 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1259 microversion: 2.1 time: 0.553561 Jul 19 11:07:16.990566 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 219/459] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:16 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1259 bytes in 555 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:17.023440 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-5ce013b0-551e-4034-9fc3-83d0fe590725 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:17.024056 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-5ce013b0-551e-4034-9fc3-83d0fe590725 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:17.055912 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=4 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:07:17.065208 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5ce013b0-551e-4034-9fc3-83d0fe590725 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:17.065660 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5ce013b0-551e-4034-9fc3-83d0fe590725 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:17.069196 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5ce013b0-551e-4034-9fc3-83d0fe590725 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:17.069687 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5ce013b0-551e-4034-9fc3-83d0fe590725 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:17.287273 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-498dd1d9-4829-41be-9e21-4fadad62d4e0 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '49e762b40bfd49859d5820d3e45adb1b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b4575dc494d40fda826dbcd0600a76a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:17.297442 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-498dd1d9-4829-41be-9e21-4fadad62d4e0 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] 10.222.0.33 "GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2" status: 200 len: 1314 microversion: 2.1 time: 0.378425 Jul 19 11:07:17.298089 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 241/460] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:16 2022] GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2 => generated 1314 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:17.453344 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-34be0ddf-e29b-4c0b-8c40-127c3a40c0ca tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:17.454127 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-34be0ddf-e29b-4c0b-8c40-127c3a40c0ca tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 988f14d7-6354-4388-a51c-b4df9749c671] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:17.463834 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-34be0ddf-e29b-4c0b-8c40-127c3a40c0ca tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:17.463834 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-34be0ddf-e29b-4c0b-8c40-127c3a40c0ca tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:17.465135 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-34be0ddf-e29b-4c0b-8c40-127c3a40c0ca tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:17.465699 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-34be0ddf-e29b-4c0b-8c40-127c3a40c0ca tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:17.578293 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-5ce013b0-551e-4034-9fc3-83d0fe590725 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:17.589466 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-5ce013b0-551e-4034-9fc3-83d0fe590725 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:17.591309 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-5ce013b0-551e-4034-9fc3-83d0fe590725 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:17.592956 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-5ce013b0-551e-4034-9fc3-83d0fe590725 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1743 microversion: 2.20 time: 0.601815 Jul 19 11:07:17.594079 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 220/461] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:16 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1743 bytes in 603 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:17.643168 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-34be0ddf-e29b-4c0b-8c40-127c3a40c0ca tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:17.655380 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-34be0ddf-e29b-4c0b-8c40-127c3a40c0ca tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671" status: 200 len: 1364 microversion: 2.1 time: 0.207340 Jul 19 11:07:17.656455 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 242/462] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:17 2022] GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671 => generated 1364 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:17.922610 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-2e32667b-e31b-4a1a-b4b6-436dcebac8fd tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:17.923211 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-2e32667b-e31b-4a1a-b4b6-436dcebac8fd tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:17.939857 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2e32667b-e31b-4a1a-b4b6-436dcebac8fd tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:17.940368 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2e32667b-e31b-4a1a-b4b6-436dcebac8fd tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:17.941331 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2e32667b-e31b-4a1a-b4b6-436dcebac8fd tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:17.941825 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2e32667b-e31b-4a1a-b4b6-436dcebac8fd tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:18.007117 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-6653dc90-10e8-4e29-9e58-d813c17965f0 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:18.007758 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-6653dc90-10e8-4e29-9e58-d813c17965f0 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:18.018780 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6653dc90-10e8-4e29-9e58-d813c17965f0 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:18.019376 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6653dc90-10e8-4e29-9e58-d813c17965f0 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:18.020383 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6653dc90-10e8-4e29-9e58-d813c17965f0 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:18.021095 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6653dc90-10e8-4e29-9e58-d813c17965f0 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:18.072629 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-2e32667b-e31b-4a1a-b4b6-436dcebac8fd tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:18.082316 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-2e32667b-e31b-4a1a-b4b6-436dcebac8fd tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1241 microversion: 2.1 time: 0.163623 Jul 19 11:07:18.087392 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 221/463] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:17 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1241 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:18.097191 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-5d072a0d-3278-4ae3-898b-08f813a0a703 req-9bb80064-6c69-412d-bf11-33ca474767d6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1c66adb2-72ed-4327-ba4c-efdef9fa18f2", "tag": "eca8d901-e37b-472e-b418-629ae81f029b"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:07:18.122567 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-5d072a0d-3278-4ae3-898b-08f813a0a703 req-9bb80064-6c69-412d-bf11-33ca474767d6 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:18.123514 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-5d072a0d-3278-4ae3-898b-08f813a0a703 req-9bb80064-6c69-412d-bf11-33ca474767d6 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:18.146592 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-5d072a0d-3278-4ae3-898b-08f813a0a703 req-9bb80064-6c69-412d-bf11-33ca474767d6 service nova] Creating event network-changed:eca8d901-e37b-472e-b418-629ae81f029b for instance 1c66adb2-72ed-4327-ba4c-efdef9fa18f2 on hv-cinder-maste Jul 19 11:07:18.157578 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-5d072a0d-3278-4ae3-898b-08f813a0a703 req-9bb80064-6c69-412d-bf11-33ca474767d6 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.068746 Jul 19 11:07:18.159129 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 222/464] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:07:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:07:18.294685 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-6653dc90-10e8-4e29-9e58-d813c17965f0 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:18.304122 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-6653dc90-10e8-4e29-9e58-d813c17965f0 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1443 microversion: 2.1 time: 0.300103 Jul 19 11:07:18.305298 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 243/465] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:18 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1443 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:07:18.315257 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-f0ce1830-1585-4f01-b426-545a93f49967 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:18.316863 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-f0ce1830-1585-4f01-b426-545a93f49967 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] [instance: 1c66adb2-72ed-4327-ba4c-efdef9fa18f2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:18.323202 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-91bc1236-f36b-4c61-a90e-9d2f7b2ebbff tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:18.324279 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-91bc1236-f36b-4c61-a90e-9d2f7b2ebbff tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:18.326362 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f0ce1830-1585-4f01-b426-545a93f49967 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:18.327074 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f0ce1830-1585-4f01-b426-545a93f49967 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:18.328721 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f0ce1830-1585-4f01-b426-545a93f49967 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:18.329375 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f0ce1830-1585-4f01-b426-545a93f49967 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:18.334727 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-91bc1236-f36b-4c61-a90e-9d2f7b2ebbff tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:18.335247 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-91bc1236-f36b-4c61-a90e-9d2f7b2ebbff tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:18.337823 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-91bc1236-f36b-4c61-a90e-9d2f7b2ebbff tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:18.338276 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-91bc1236-f36b-4c61-a90e-9d2f7b2ebbff tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:18.481752 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-91bc1236-f36b-4c61-a90e-9d2f7b2ebbff tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:18.489363 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-91bc1236-f36b-4c61-a90e-9d2f7b2ebbff tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1443 microversion: 2.1 time: 0.172282 Jul 19 11:07:18.489933 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 244/466] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:18 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1443 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:07:18.577498 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-f0ce1830-1585-4f01-b426-545a93f49967 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '49e762b40bfd49859d5820d3e45adb1b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b4575dc494d40fda826dbcd0600a76a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:18.585113 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-f0ce1830-1585-4f01-b426-545a93f49967 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] 10.222.0.33 "GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2" status: 200 len: 1314 microversion: 2.1 time: 0.274250 Jul 19 11:07:18.594847 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 223/467] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:18 2022] GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2 => generated 1314 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:18.611286 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-0d67495f-55f3-4040-b43b-3074f5cb2f08 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:18.611885 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-0d67495f-55f3-4040-b43b-3074f5cb2f08 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:18.619346 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0d67495f-55f3-4040-b43b-3074f5cb2f08 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:18.620015 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0d67495f-55f3-4040-b43b-3074f5cb2f08 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:18.621059 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0d67495f-55f3-4040-b43b-3074f5cb2f08 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:18.621643 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0d67495f-55f3-4040-b43b-3074f5cb2f08 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:18.672741 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-25315957-685f-4b32-bcb8-d4c72c5b551e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:18.673338 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-25315957-685f-4b32-bcb8-d4c72c5b551e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 988f14d7-6354-4388-a51c-b4df9749c671] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:18.689203 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-25315957-685f-4b32-bcb8-d4c72c5b551e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:18.689959 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-25315957-685f-4b32-bcb8-d4c72c5b551e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:18.698010 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-25315957-685f-4b32-bcb8-d4c72c5b551e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:18.698671 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-25315957-685f-4b32-bcb8-d4c72c5b551e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:18.870947 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-0d67495f-55f3-4040-b43b-3074f5cb2f08 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:18.882454 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-0d67495f-55f3-4040-b43b-3074f5cb2f08 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:18.884229 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-0d67495f-55f3-4040-b43b-3074f5cb2f08 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:18.885603 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-0d67495f-55f3-4040-b43b-3074f5cb2f08 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1743 microversion: 2.20 time: 0.278214 Jul 19 11:07:18.896890 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 245/468] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:18 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1743 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:19.022749 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-25315957-685f-4b32-bcb8-d4c72c5b551e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:19.031343 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-25315957-685f-4b32-bcb8-d4c72c5b551e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671" status: 200 len: 1364 microversion: 2.1 time: 0.362305 Jul 19 11:07:19.032360 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 224/469] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:18 2022] GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671 => generated 1364 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:07:19.102444 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-0ee271a3-fe73-46fd-8127-83d20813e802 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:19.103042 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-0ee271a3-fe73-46fd-8127-83d20813e802 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:19.116513 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0ee271a3-fe73-46fd-8127-83d20813e802 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:19.116999 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0ee271a3-fe73-46fd-8127-83d20813e802 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:19.117851 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0ee271a3-fe73-46fd-8127-83d20813e802 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:19.118210 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0ee271a3-fe73-46fd-8127-83d20813e802 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:19.317292 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-0ee271a3-fe73-46fd-8127-83d20813e802 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:19.328719 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-0ee271a3-fe73-46fd-8127-83d20813e802 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1303 microversion: 2.1 time: 0.230056 Jul 19 11:07:19.330066 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 246/470] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:19 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1303 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:19.602801 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-a1e44ec1-d1d6-45fe-bdd3-ef2adbe9702a tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:19.605041 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-a1e44ec1-d1d6-45fe-bdd3-ef2adbe9702a tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] [instance: 1c66adb2-72ed-4327-ba4c-efdef9fa18f2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:19.613339 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a1e44ec1-d1d6-45fe-bdd3-ef2adbe9702a tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:19.614076 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a1e44ec1-d1d6-45fe-bdd3-ef2adbe9702a tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:19.615241 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a1e44ec1-d1d6-45fe-bdd3-ef2adbe9702a tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:19.615834 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a1e44ec1-d1d6-45fe-bdd3-ef2adbe9702a tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:19.827457 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-9043c316-44d2-4c4c-97a7-2d78fa8cb1ac req-fdbb71e8-93cb-41e8-88ef-9011e3a772f9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "988f14d7-6354-4388-a51c-b4df9749c671", "name": "network-vif-plugged", "status": "completed", "tag": "ebf84be3-6be9-46a9-b6fd-10aca9db5a6f"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:07:19.836084 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-9043c316-44d2-4c4c-97a7-2d78fa8cb1ac req-fdbb71e8-93cb-41e8-88ef-9011e3a772f9 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:19.836892 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-9043c316-44d2-4c4c-97a7-2d78fa8cb1ac req-fdbb71e8-93cb-41e8-88ef-9011e3a772f9 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:19.849978 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-9043c316-44d2-4c4c-97a7-2d78fa8cb1ac req-fdbb71e8-93cb-41e8-88ef-9011e3a772f9 service nova] Creating event network-vif-plugged:ebf84be3-6be9-46a9-b6fd-10aca9db5a6f for instance 988f14d7-6354-4388-a51c-b4df9749c671 on hv-cinder-maste Jul 19 11:07:19.856239 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-9043c316-44d2-4c4c-97a7-2d78fa8cb1ac req-fdbb71e8-93cb-41e8-88ef-9011e3a772f9 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032350 Jul 19 11:07:19.857441 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 247/471] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:07:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:07:19.885851 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-a1e44ec1-d1d6-45fe-bdd3-ef2adbe9702a tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '49e762b40bfd49859d5820d3e45adb1b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b4575dc494d40fda826dbcd0600a76a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:19.894527 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-a1e44ec1-d1d6-45fe-bdd3-ef2adbe9702a tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] 10.222.0.33 "GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2" status: 200 len: 1314 microversion: 2.1 time: 0.296644 Jul 19 11:07:19.895368 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 225/472] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:19 2022] GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2 => generated 1314 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:19.908979 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-a63e2724-62f5-41b8-9829-a4b3f9f2e4ba tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:19.910554 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-a63e2724-62f5-41b8-9829-a4b3f9f2e4ba tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:19.919249 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a63e2724-62f5-41b8-9829-a4b3f9f2e4ba tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:19.919960 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a63e2724-62f5-41b8-9829-a4b3f9f2e4ba tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:19.921150 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a63e2724-62f5-41b8-9829-a4b3f9f2e4ba tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:19.921792 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a63e2724-62f5-41b8-9829-a4b3f9f2e4ba tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:20.048796 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-3c055e71-bc7a-4f02-9b78-fa734a105524 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:20.049672 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-3c055e71-bc7a-4f02-9b78-fa734a105524 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 988f14d7-6354-4388-a51c-b4df9749c671] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:20.057262 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3c055e71-bc7a-4f02-9b78-fa734a105524 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:20.057947 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3c055e71-bc7a-4f02-9b78-fa734a105524 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:20.058914 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3c055e71-bc7a-4f02-9b78-fa734a105524 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:20.059506 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3c055e71-bc7a-4f02-9b78-fa734a105524 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:20.275228 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-a63e2724-62f5-41b8-9829-a4b3f9f2e4ba tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:20.287477 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-a63e2724-62f5-41b8-9829-a4b3f9f2e4ba tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:20.294631 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-a63e2724-62f5-41b8-9829-a4b3f9f2e4ba tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:20.296205 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-a63e2724-62f5-41b8-9829-a4b3f9f2e4ba tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1738 microversion: 2.20 time: 0.396132 Jul 19 11:07:20.300720 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 248/473] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:19 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1738 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:07:20.316144 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-3c055e71-bc7a-4f02-9b78-fa734a105524 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:20.323856 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-3c055e71-bc7a-4f02-9b78-fa734a105524 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671" status: 200 len: 1364 microversion: 2.1 time: 0.278762 Jul 19 11:07:20.324440 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 226/474] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:20 2022] GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671 => generated 1364 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:07:20.347016 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-4a8227bb-265d-4d3f-abd3-1df303326f52 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:20.347737 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-4a8227bb-265d-4d3f-abd3-1df303326f52 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:20.354790 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4a8227bb-265d-4d3f-abd3-1df303326f52 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:20.355571 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4a8227bb-265d-4d3f-abd3-1df303326f52 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:20.361433 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4a8227bb-265d-4d3f-abd3-1df303326f52 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:20.362234 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4a8227bb-265d-4d3f-abd3-1df303326f52 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:20.617491 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-4a8227bb-265d-4d3f-abd3-1df303326f52 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:20.633736 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-4a8227bb-265d-4d3f-abd3-1df303326f52 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1303 microversion: 2.1 time: 0.289791 Jul 19 11:07:20.635094 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 249/475] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:20 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1303 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:20.916551 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-6e5866a4-1eba-47e5-a844-7d26f1b94746 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:20.918631 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-6e5866a4-1eba-47e5-a844-7d26f1b94746 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] [instance: 1c66adb2-72ed-4327-ba4c-efdef9fa18f2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:20.922241 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell0:SELECT=2 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:07:20.931186 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6e5866a4-1eba-47e5-a844-7d26f1b94746 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:20.931916 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6e5866a4-1eba-47e5-a844-7d26f1b94746 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:20.933550 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6e5866a4-1eba-47e5-a844-7d26f1b94746 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:20.934321 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6e5866a4-1eba-47e5-a844-7d26f1b94746 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:21.319352 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-28339f13-2cfb-4836-829f-132f176ed19d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:21.320059 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-28339f13-2cfb-4836-829f-132f176ed19d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:21.338568 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-28339f13-2cfb-4836-829f-132f176ed19d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:21.339020 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-28339f13-2cfb-4836-829f-132f176ed19d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:21.347461 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-28339f13-2cfb-4836-829f-132f176ed19d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:21.348538 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-28339f13-2cfb-4836-829f-132f176ed19d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:21.618353 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-6e5866a4-1eba-47e5-a844-7d26f1b94746 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '49e762b40bfd49859d5820d3e45adb1b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b4575dc494d40fda826dbcd0600a76a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:21.634850 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-6e5866a4-1eba-47e5-a844-7d26f1b94746 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] 10.222.0.33 "GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2" status: 200 len: 1314 microversion: 2.1 time: 0.727050 Jul 19 11:07:21.635465 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 227/476] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:20 2022] GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2 => generated 1314 bytes in 729 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:21.643217 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-80e8a54b-8d3b-4009-90f4-71a2a09d2a0c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:21.644169 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-80e8a54b-8d3b-4009-90f4-71a2a09d2a0c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 988f14d7-6354-4388-a51c-b4df9749c671] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:21.674222 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-80e8a54b-8d3b-4009-90f4-71a2a09d2a0c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:21.675539 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-80e8a54b-8d3b-4009-90f4-71a2a09d2a0c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:21.676813 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-80e8a54b-8d3b-4009-90f4-71a2a09d2a0c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:21.677494 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-80e8a54b-8d3b-4009-90f4-71a2a09d2a0c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:21.860013 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-28339f13-2cfb-4836-829f-132f176ed19d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:21.879763 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-28339f13-2cfb-4836-829f-132f176ed19d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:21.881328 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-28339f13-2cfb-4836-829f-132f176ed19d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:21.882508 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-28339f13-2cfb-4836-829f-132f176ed19d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1738 microversion: 2.20 time: 0.570786 Jul 19 11:07:21.893749 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 250/477] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:21 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1738 bytes in 583 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:07:21.899547 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-e02ba61e-7dd6-4bf4-991b-44f0b485830b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:21.900400 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-e02ba61e-7dd6-4bf4-991b-44f0b485830b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:21.947063 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e02ba61e-7dd6-4bf4-991b-44f0b485830b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:21.947724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e02ba61e-7dd6-4bf4-991b-44f0b485830b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:21.948727 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e02ba61e-7dd6-4bf4-991b-44f0b485830b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:21.949332 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e02ba61e-7dd6-4bf4-991b-44f0b485830b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:21.994057 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-80e8a54b-8d3b-4009-90f4-71a2a09d2a0c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:22.007453 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-80e8a54b-8d3b-4009-90f4-71a2a09d2a0c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671" status: 200 len: 1364 microversion: 2.1 time: 0.369581 Jul 19 11:07:22.008845 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 228/478] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:21 2022] GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671 => generated 1364 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:22.032301 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-052953ee-4c36-4abf-94b6-76f35ff337db req-0b64f074-e6ec-44e5-a33c-bc09a1a69415 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f52fbfae-577b-403d-a4d3-9a4b79ae2326", "name": "network-vif-plugged", "status": "completed", "tag": "748a7315-79e9-4ab8-ad6a-336c057e8b2c"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:07:22.042815 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-052953ee-4c36-4abf-94b6-76f35ff337db req-0b64f074-e6ec-44e5-a33c-bc09a1a69415 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:22.044031 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-052953ee-4c36-4abf-94b6-76f35ff337db req-0b64f074-e6ec-44e5-a33c-bc09a1a69415 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:22.069275 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-052953ee-4c36-4abf-94b6-76f35ff337db req-0b64f074-e6ec-44e5-a33c-bc09a1a69415 service nova] Creating event network-vif-plugged:748a7315-79e9-4ab8-ad6a-336c057e8b2c for instance f52fbfae-577b-403d-a4d3-9a4b79ae2326 on hv-cinder-maste Jul 19 11:07:22.086742 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-052953ee-4c36-4abf-94b6-76f35ff337db req-0b64f074-e6ec-44e5-a33c-bc09a1a69415 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.074352 Jul 19 11:07:22.088759 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 229/479] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:07:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:07:22.652481 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-45fd28ad-be27-47cc-949d-1b572000f6a7 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:22.653701 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-45fd28ad-be27-47cc-949d-1b572000f6a7 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] [instance: 1c66adb2-72ed-4327-ba4c-efdef9fa18f2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:22.662097 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-45fd28ad-be27-47cc-949d-1b572000f6a7 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:22.664738 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-45fd28ad-be27-47cc-949d-1b572000f6a7 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:22.671557 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-45fd28ad-be27-47cc-949d-1b572000f6a7 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:22.674297 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-45fd28ad-be27-47cc-949d-1b572000f6a7 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:22.882537 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-e02ba61e-7dd6-4bf4-991b-44f0b485830b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:22.892091 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-e02ba61e-7dd6-4bf4-991b-44f0b485830b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1370 microversion: 2.1 time: 0.996833 Jul 19 11:07:22.901068 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell0:SELECT=10 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:07:22.910835 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 251/480] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:21 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1370 bytes in 1016 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:22.928559 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-cd1a2dae-39d0-403d-b8d9-7620990f0f3a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:22.959476 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-cd1a2dae-39d0-403d-b8d9-7620990f0f3a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:22.984883 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cd1a2dae-39d0-403d-b8d9-7620990f0f3a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:22.985344 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cd1a2dae-39d0-403d-b8d9-7620990f0f3a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:22.986116 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cd1a2dae-39d0-403d-b8d9-7620990f0f3a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:22.989693 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cd1a2dae-39d0-403d-b8d9-7620990f0f3a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:23.013373 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-45fd28ad-be27-47cc-949d-1b572000f6a7 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '49e762b40bfd49859d5820d3e45adb1b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b4575dc494d40fda826dbcd0600a76a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:23.025967 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-45fd28ad-be27-47cc-949d-1b572000f6a7 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] 10.222.0.33 "GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2" status: 200 len: 1314 microversion: 2.1 time: 0.378196 Jul 19 11:07:23.027619 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 230/481] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:22 2022] GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2 => generated 1314 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:07:23.065066 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-9a280781-acbc-4873-b66d-2464b210e938 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:23.065995 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-9a280781-acbc-4873-b66d-2464b210e938 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 988f14d7-6354-4388-a51c-b4df9749c671] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:23.074331 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9a280781-acbc-4873-b66d-2464b210e938 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:23.075220 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9a280781-acbc-4873-b66d-2464b210e938 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:23.076225 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9a280781-acbc-4873-b66d-2464b210e938 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:23.079812 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9a280781-acbc-4873-b66d-2464b210e938 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:23.671988 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-9a280781-acbc-4873-b66d-2464b210e938 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:23.672670 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-cd1a2dae-39d0-403d-b8d9-7620990f0f3a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:23.688174 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-9a280781-acbc-4873-b66d-2464b210e938 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671" status: 200 len: 1364 microversion: 2.1 time: 0.658048 Jul 19 11:07:23.691314 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-cd1a2dae-39d0-403d-b8d9-7620990f0f3a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:23.700933 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 231/482] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:23 2022] GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671 => generated 1364 bytes in 662 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:23.703198 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-cd1a2dae-39d0-403d-b8d9-7620990f0f3a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:23.706698 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-cd1a2dae-39d0-403d-b8d9-7620990f0f3a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1738 microversion: 2.20 time: 0.794776 Jul 19 11:07:23.712210 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 252/483] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:22 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1738 bytes in 801 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:23.917747 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-bf0c1df0-26db-468c-9096-459090488679 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:23.918373 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-bf0c1df0-26db-468c-9096-459090488679 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:23.935920 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bf0c1df0-26db-468c-9096-459090488679 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:23.936365 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bf0c1df0-26db-468c-9096-459090488679 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:23.942410 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bf0c1df0-26db-468c-9096-459090488679 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:23.943124 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bf0c1df0-26db-468c-9096-459090488679 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:24.024897 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-28cbe2f4-e62b-48f3-99a0-c96b8be97e74 req-198a50ae-6024-46d5-80d7-3520551dbfff service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5b72a14b-bd18-4dfa-a68f-12a5424f471c", "tag": "da843419-4401-4a26-82c2-7930b51aa1b8"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:07:24.036557 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-28cbe2f4-e62b-48f3-99a0-c96b8be97e74 req-198a50ae-6024-46d5-80d7-3520551dbfff service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:24.037424 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-28cbe2f4-e62b-48f3-99a0-c96b8be97e74 req-198a50ae-6024-46d5-80d7-3520551dbfff service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:24.069732 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-28cbe2f4-e62b-48f3-99a0-c96b8be97e74 req-198a50ae-6024-46d5-80d7-3520551dbfff service nova] Creating event network-changed:da843419-4401-4a26-82c2-7930b51aa1b8 for instance 5b72a14b-bd18-4dfa-a68f-12a5424f471c on hv-cinder-maste Jul 19 11:07:24.085095 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-28cbe2f4-e62b-48f3-99a0-c96b8be97e74 req-198a50ae-6024-46d5-80d7-3520551dbfff service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.061374 Jul 19 11:07:24.086598 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 253/484] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:07:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:07:24.100116 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-5ba100c1-a0af-44d2-bb2f-149bce572767 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:24.103472 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-5ba100c1-a0af-44d2-bb2f-149bce572767 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] [instance: 1c66adb2-72ed-4327-ba4c-efdef9fa18f2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:24.119659 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5ba100c1-a0af-44d2-bb2f-149bce572767 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:24.120101 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5ba100c1-a0af-44d2-bb2f-149bce572767 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:24.122040 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5ba100c1-a0af-44d2-bb2f-149bce572767 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:24.126090 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5ba100c1-a0af-44d2-bb2f-149bce572767 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:24.439552 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-5ba100c1-a0af-44d2-bb2f-149bce572767 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '49e762b40bfd49859d5820d3e45adb1b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b4575dc494d40fda826dbcd0600a76a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:24.456242 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-5ba100c1-a0af-44d2-bb2f-149bce572767 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] 10.222.0.33 "GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2" status: 200 len: 1314 microversion: 2.1 time: 0.368109 Jul 19 11:07:24.502415 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 254/485] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:24 2022] GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2 => generated 1314 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:24.526311 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-bf0c1df0-26db-468c-9096-459090488679 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:24.536578 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-bf0c1df0-26db-468c-9096-459090488679 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1370 microversion: 2.1 time: 0.626672 Jul 19 11:07:24.560062 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 232/486] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:23 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1370 bytes in 651 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:24.709399 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-78d150c2-84ff-4679-b7bc-dd95d32093d0 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:24.710094 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-78d150c2-84ff-4679-b7bc-dd95d32093d0 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 988f14d7-6354-4388-a51c-b4df9749c671] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:24.731754 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-a0516c8e-a896-43bc-b55b-3de55be52615 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:24.733319 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-a0516c8e-a896-43bc-b55b-3de55be52615 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:24.745867 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a0516c8e-a896-43bc-b55b-3de55be52615 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:24.746860 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-78d150c2-84ff-4679-b7bc-dd95d32093d0 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:24.747697 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-78d150c2-84ff-4679-b7bc-dd95d32093d0 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.024s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:24.748913 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-78d150c2-84ff-4679-b7bc-dd95d32093d0 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:24.750417 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-78d150c2-84ff-4679-b7bc-dd95d32093d0 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:24.754329 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a0516c8e-a896-43bc-b55b-3de55be52615 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:24.756178 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a0516c8e-a896-43bc-b55b-3de55be52615 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:24.760040 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a0516c8e-a896-43bc-b55b-3de55be52615 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:25.239754 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-a0516c8e-a896-43bc-b55b-3de55be52615 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:25.241808 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-78d150c2-84ff-4679-b7bc-dd95d32093d0 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:25.251116 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-78d150c2-84ff-4679-b7bc-dd95d32093d0 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671" status: 200 len: 1555 microversion: 2.1 time: 0.545644 Jul 19 11:07:25.254631 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 255/487] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:24 2022] GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671 => generated 1555 bytes in 550 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:25.255668 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-a0516c8e-a896-43bc-b55b-3de55be52615 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:25.258277 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-a0516c8e-a896-43bc-b55b-3de55be52615 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:25.259482 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-a0516c8e-a896-43bc-b55b-3de55be52615 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1738 microversion: 2.20 time: 0.533553 Jul 19 11:07:25.260032 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 233/488] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:24 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1738 bytes in 535 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:25.283935 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-0c96c2a6-948f-4522-978c-359fe7cb075f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:25.284735 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-0c96c2a6-948f-4522-978c-359fe7cb075f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 988f14d7-6354-4388-a51c-b4df9749c671] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:25.295402 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0c96c2a6-948f-4522-978c-359fe7cb075f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:25.296109 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0c96c2a6-948f-4522-978c-359fe7cb075f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:25.297091 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0c96c2a6-948f-4522-978c-359fe7cb075f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:25.299003 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0c96c2a6-948f-4522-978c-359fe7cb075f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:25.482676 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-5500bbb6-53aa-4d6f-95d0-0f67f1da3563 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:25.483281 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-5500bbb6-53aa-4d6f-95d0-0f67f1da3563 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] [instance: 1c66adb2-72ed-4327-ba4c-efdef9fa18f2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:25.506844 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5500bbb6-53aa-4d6f-95d0-0f67f1da3563 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:25.507608 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5500bbb6-53aa-4d6f-95d0-0f67f1da3563 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:25.509183 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5500bbb6-53aa-4d6f-95d0-0f67f1da3563 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:25.510327 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5500bbb6-53aa-4d6f-95d0-0f67f1da3563 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:25.625529 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-0c96c2a6-948f-4522-978c-359fe7cb075f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.345131 Jul 19 11:07:25.626495 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 256/489] 10.222.0.33 () {58 vars in 1274 bytes} [Tue Jul 19 11:07:25 2022] GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671/os-interface => generated 285 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:07:25.638015 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-0c779c1e-7f90-495e-a5aa-3b783fa0de35 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:25.638800 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-0c779c1e-7f90-495e-a5aa-3b783fa0de35 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:25.652928 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0c779c1e-7f90-495e-a5aa-3b783fa0de35 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:25.655977 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0c779c1e-7f90-495e-a5aa-3b783fa0de35 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:25.657200 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0c779c1e-7f90-495e-a5aa-3b783fa0de35 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:25.660090 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0c779c1e-7f90-495e-a5aa-3b783fa0de35 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:25.718565 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=100 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:07:25.801005 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-5500bbb6-53aa-4d6f-95d0-0f67f1da3563 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '49e762b40bfd49859d5820d3e45adb1b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b4575dc494d40fda826dbcd0600a76a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:25.816043 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-5500bbb6-53aa-4d6f-95d0-0f67f1da3563 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] 10.222.0.33 "GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2" status: 200 len: 1314 microversion: 2.1 time: 0.337542 Jul 19 11:07:25.818419 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 234/490] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:25 2022] GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2 => generated 1314 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:26.129371 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-07849a2a-6e10-475a-b989-1af19e8b6bf3 req-9f3194ae-7150-430b-94bd-f08035a508bf service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b9897cfb-9ec6-41f4-bcb5-ba1a978421d3", "tag": "bee94173-a17a-45b4-ab19-e862338cf16b"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:07:26.160294 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-07849a2a-6e10-475a-b989-1af19e8b6bf3 req-9f3194ae-7150-430b-94bd-f08035a508bf service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:26.163457 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-07849a2a-6e10-475a-b989-1af19e8b6bf3 req-9f3194ae-7150-430b-94bd-f08035a508bf service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:26.201390 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-07849a2a-6e10-475a-b989-1af19e8b6bf3 req-9f3194ae-7150-430b-94bd-f08035a508bf service nova] Creating event network-changed:bee94173-a17a-45b4-ab19-e862338cf16b for instance b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 on hv-cinder-maste Jul 19 11:07:26.215304 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-07849a2a-6e10-475a-b989-1af19e8b6bf3 req-9f3194ae-7150-430b-94bd-f08035a508bf service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.087770 Jul 19 11:07:26.216336 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 235/491] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:07:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:07:26.228063 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-0c779c1e-7f90-495e-a5aa-3b783fa0de35 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:26.245853 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-0c779c1e-7f90-495e-a5aa-3b783fa0de35 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1370 microversion: 2.1 time: 0.611771 Jul 19 11:07:26.246546 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 257/492] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:25 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1370 bytes in 620 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:26.290261 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-db8b8157-b428-40ff-a3d7-247189fe7ed3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:26.291558 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-db8b8157-b428-40ff-a3d7-247189fe7ed3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:26.320504 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-db8b8157-b428-40ff-a3d7-247189fe7ed3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:26.323603 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-db8b8157-b428-40ff-a3d7-247189fe7ed3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:26.324792 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-db8b8157-b428-40ff-a3d7-247189fe7ed3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:26.325245 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-db8b8157-b428-40ff-a3d7-247189fe7ed3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:26.712497 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-db8b8157-b428-40ff-a3d7-247189fe7ed3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:26.716347 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=100 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:07:26.722837 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-db8b8157-b428-40ff-a3d7-247189fe7ed3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:26.727560 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-db8b8157-b428-40ff-a3d7-247189fe7ed3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:26.732107 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-db8b8157-b428-40ff-a3d7-247189fe7ed3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1738 microversion: 2.20 time: 0.449733 Jul 19 11:07:26.735508 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 236/493] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:26 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1738 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:26.835573 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-50b2209a-a934-40ff-a912-2b6c79561d30 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:26.836510 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-50b2209a-a934-40ff-a912-2b6c79561d30 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] [instance: 1c66adb2-72ed-4327-ba4c-efdef9fa18f2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:26.838977 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell0:SELECT=2 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:07:26.849784 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-50b2209a-a934-40ff-a912-2b6c79561d30 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:26.855215 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-50b2209a-a934-40ff-a912-2b6c79561d30 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:26.856420 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-50b2209a-a934-40ff-a912-2b6c79561d30 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:26.857499 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-50b2209a-a934-40ff-a912-2b6c79561d30 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:26.907022 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell0:SELECT=22 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:07:27.264658 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-aeffa73b-3a53-4ce9-93ab-19bf9e5fdb85 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:27.268290 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-aeffa73b-3a53-4ce9-93ab-19bf9e5fdb85 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:27.291693 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-aeffa73b-3a53-4ce9-93ab-19bf9e5fdb85 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:27.305906 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-aeffa73b-3a53-4ce9-93ab-19bf9e5fdb85 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:27.317844 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-aeffa73b-3a53-4ce9-93ab-19bf9e5fdb85 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:27.318660 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-aeffa73b-3a53-4ce9-93ab-19bf9e5fdb85 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:27.727955 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-50b2209a-a934-40ff-a912-2b6c79561d30 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '49e762b40bfd49859d5820d3e45adb1b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b4575dc494d40fda826dbcd0600a76a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:27.741768 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-50b2209a-a934-40ff-a912-2b6c79561d30 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] 10.222.0.33 "GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2" status: 200 len: 1314 microversion: 2.1 time: 0.909827 Jul 19 11:07:27.750146 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-aeffa73b-3a53-4ce9-93ab-19bf9e5fdb85 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:27.759129 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 258/494] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:26 2022] GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2 => generated 1314 bytes in 928 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:27.761258 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-aeffa73b-3a53-4ce9-93ab-19bf9e5fdb85 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1370 microversion: 2.1 time: 0.501608 Jul 19 11:07:27.765236 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 237/495] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:27 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1370 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:07:27.778451 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-4d06756f-b195-4a1f-8e38-442ccbf35387 req-72250112-28bb-4f1d-b0ca-9ffeec56f9bc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1c66adb2-72ed-4327-ba4c-efdef9fa18f2", "name": "network-vif-plugged", "status": "completed", "tag": "eca8d901-e37b-472e-b418-629ae81f029b"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:07:27.779578 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-b1602f12-21ac-4916-ae2e-efd5ff5a4a07 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:27.782317 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-b1602f12-21ac-4916-ae2e-efd5ff5a4a07 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:27.803725 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b1602f12-21ac-4916-ae2e-efd5ff5a4a07 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:27.805881 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b1602f12-21ac-4916-ae2e-efd5ff5a4a07 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:27.807979 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b1602f12-21ac-4916-ae2e-efd5ff5a4a07 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:27.808585 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b1602f12-21ac-4916-ae2e-efd5ff5a4a07 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:27.810621 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-4d06756f-b195-4a1f-8e38-442ccbf35387 req-72250112-28bb-4f1d-b0ca-9ffeec56f9bc service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:27.816106 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-4d06756f-b195-4a1f-8e38-442ccbf35387 req-72250112-28bb-4f1d-b0ca-9ffeec56f9bc service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:27.847213 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-4d06756f-b195-4a1f-8e38-442ccbf35387 req-72250112-28bb-4f1d-b0ca-9ffeec56f9bc service nova] Creating event network-vif-plugged:eca8d901-e37b-472e-b418-629ae81f029b for instance 1c66adb2-72ed-4327-ba4c-efdef9fa18f2 on hv-cinder-maste Jul 19 11:07:27.861969 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-4d06756f-b195-4a1f-8e38-442ccbf35387 req-72250112-28bb-4f1d-b0ca-9ffeec56f9bc service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.099228 Jul 19 11:07:27.864853 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 259/496] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:07:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:07:28.776304 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-9eb409ab-cb11-453d-b9ab-4b0af1cc0d94 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:28.777051 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-9eb409ab-cb11-453d-b9ab-4b0af1cc0d94 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] [instance: 1c66adb2-72ed-4327-ba4c-efdef9fa18f2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:28.794509 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9eb409ab-cb11-453d-b9ab-4b0af1cc0d94 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:28.798004 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9eb409ab-cb11-453d-b9ab-4b0af1cc0d94 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:28.798897 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9eb409ab-cb11-453d-b9ab-4b0af1cc0d94 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:28.800567 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9eb409ab-cb11-453d-b9ab-4b0af1cc0d94 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:28.976562 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b1602f12-21ac-4916-ae2e-efd5ff5a4a07 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:28.993308 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b1602f12-21ac-4916-ae2e-efd5ff5a4a07 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:28.998914 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b1602f12-21ac-4916-ae2e-efd5ff5a4a07 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:29.004248 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-b1602f12-21ac-4916-ae2e-efd5ff5a4a07 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1738 microversion: 2.20 time: 1.237761 Jul 19 11:07:29.016437 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 238/497] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:27 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1738 bytes in 1245 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:29.038749 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-e41b36f4-808c-4a0c-99cc-c67dda8011c7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:29.063735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-e41b36f4-808c-4a0c-99cc-c67dda8011c7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:29.074698 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e41b36f4-808c-4a0c-99cc-c67dda8011c7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:29.093310 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e41b36f4-808c-4a0c-99cc-c67dda8011c7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.019s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:29.098951 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e41b36f4-808c-4a0c-99cc-c67dda8011c7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:29.099603 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e41b36f4-808c-4a0c-99cc-c67dda8011c7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:29.218119 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-9eb409ab-cb11-453d-b9ab-4b0af1cc0d94 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '49e762b40bfd49859d5820d3e45adb1b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b4575dc494d40fda826dbcd0600a76a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:29.230943 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-9eb409ab-cb11-453d-b9ab-4b0af1cc0d94 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] 10.222.0.33 "GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2" status: 200 len: 1314 microversion: 2.1 time: 0.461779 Jul 19 11:07:29.240606 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 260/498] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:28 2022] GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2 => generated 1314 bytes in 476 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:29.253208 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-47b46bf8-5eb4-4a0e-8b61-2df9fcd1f7b1 req-6c287f7f-31ef-4389-b7ea-8f70f990a560 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "988f14d7-6354-4388-a51c-b4df9749c671", "tag": "ebf84be3-6be9-46a9-b6fd-10aca9db5a6f"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:07:29.263412 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-47b46bf8-5eb4-4a0e-8b61-2df9fcd1f7b1 req-6c287f7f-31ef-4389-b7ea-8f70f990a560 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:29.264384 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-47b46bf8-5eb4-4a0e-8b61-2df9fcd1f7b1 req-6c287f7f-31ef-4389-b7ea-8f70f990a560 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:29.278405 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-47b46bf8-5eb4-4a0e-8b61-2df9fcd1f7b1 req-6c287f7f-31ef-4389-b7ea-8f70f990a560 service nova] Creating event network-changed:ebf84be3-6be9-46a9-b6fd-10aca9db5a6f for instance 988f14d7-6354-4388-a51c-b4df9749c671 on hv-cinder-maste Jul 19 11:07:29.284175 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-47b46bf8-5eb4-4a0e-8b61-2df9fcd1f7b1 req-6c287f7f-31ef-4389-b7ea-8f70f990a560 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042174 Jul 19 11:07:29.289443 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 261/499] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:07:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:07:29.449616 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-e41b36f4-808c-4a0c-99cc-c67dda8011c7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:29.460569 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-e41b36f4-808c-4a0c-99cc-c67dda8011c7 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1370 microversion: 2.1 time: 0.447822 Jul 19 11:07:29.461562 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 239/500] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:29 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1370 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:30.024402 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-fa4d724f-60f2-4474-9eb5-ada9af56a77b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:30.025821 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-fa4d724f-60f2-4474-9eb5-ada9af56a77b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:30.036640 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fa4d724f-60f2-4474-9eb5-ada9af56a77b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:30.043074 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fa4d724f-60f2-4474-9eb5-ada9af56a77b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:30.044157 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fa4d724f-60f2-4474-9eb5-ada9af56a77b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:30.044805 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fa4d724f-60f2-4474-9eb5-ada9af56a77b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:30.254912 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-b7878cf2-6aad-4d4f-b19a-ec8263ec7aba tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:30.256114 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-b7878cf2-6aad-4d4f-b19a-ec8263ec7aba tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] [instance: 1c66adb2-72ed-4327-ba4c-efdef9fa18f2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:30.265844 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b7878cf2-6aad-4d4f-b19a-ec8263ec7aba tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:30.266854 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b7878cf2-6aad-4d4f-b19a-ec8263ec7aba tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:30.268511 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b7878cf2-6aad-4d4f-b19a-ec8263ec7aba tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:30.269371 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b7878cf2-6aad-4d4f-b19a-ec8263ec7aba tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:30.412232 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-fa4d724f-60f2-4474-9eb5-ada9af56a77b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:30.425309 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-fa4d724f-60f2-4474-9eb5-ada9af56a77b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:30.426829 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-fa4d724f-60f2-4474-9eb5-ada9af56a77b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:30.428005 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-fa4d724f-60f2-4474-9eb5-ada9af56a77b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1738 microversion: 2.20 time: 0.407810 Jul 19 11:07:30.450340 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 262/501] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:30 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1738 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:30.478237 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-f65e03e0-124a-4791-83cb-7cb71c299c46 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:30.479129 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-f65e03e0-124a-4791-83cb-7cb71c299c46 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:30.499141 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f65e03e0-124a-4791-83cb-7cb71c299c46 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:30.500340 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f65e03e0-124a-4791-83cb-7cb71c299c46 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:30.501473 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f65e03e0-124a-4791-83cb-7cb71c299c46 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:30.502002 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f65e03e0-124a-4791-83cb-7cb71c299c46 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:30.833996 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-f65e03e0-124a-4791-83cb-7cb71c299c46 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:30.843702 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-f65e03e0-124a-4791-83cb-7cb71c299c46 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1370 microversion: 2.1 time: 0.369759 Jul 19 11:07:30.847302 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 263/502] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:30 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1370 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:30.871449 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b7878cf2-6aad-4d4f-b19a-ec8263ec7aba tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '49e762b40bfd49859d5820d3e45adb1b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b4575dc494d40fda826dbcd0600a76a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:30.882070 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-b7878cf2-6aad-4d4f-b19a-ec8263ec7aba tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] 10.222.0.33 "GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2" status: 200 len: 1314 microversion: 2.1 time: 0.637445 Jul 19 11:07:30.883269 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 240/503] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:30 2022] GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2 => generated 1314 bytes in 639 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:31.456392 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-0e97f552-c1e3-41c9-aecc-508cb64ff6ff tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:31.458139 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-0e97f552-c1e3-41c9-aecc-508cb64ff6ff tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:31.473584 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0e97f552-c1e3-41c9-aecc-508cb64ff6ff tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:31.478383 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0e97f552-c1e3-41c9-aecc-508cb64ff6ff tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:31.479337 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0e97f552-c1e3-41c9-aecc-508cb64ff6ff tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:31.482965 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0e97f552-c1e3-41c9-aecc-508cb64ff6ff tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:31.868258 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-966275e7-3fff-456f-a82d-cf0c00431e00 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:31.869081 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-966275e7-3fff-456f-a82d-cf0c00431e00 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:31.880585 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-966275e7-3fff-456f-a82d-cf0c00431e00 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:31.883072 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-966275e7-3fff-456f-a82d-cf0c00431e00 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:31.884100 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-966275e7-3fff-456f-a82d-cf0c00431e00 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:31.884602 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-966275e7-3fff-456f-a82d-cf0c00431e00 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:31.987102 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-0e97f552-c1e3-41c9-aecc-508cb64ff6ff tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:32.000212 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-0e97f552-c1e3-41c9-aecc-508cb64ff6ff tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:32.001771 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-0e97f552-c1e3-41c9-aecc-508cb64ff6ff tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:32.003011 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-0e97f552-c1e3-41c9-aecc-508cb64ff6ff tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1738 microversion: 2.20 time: 0.550526 Jul 19 11:07:32.003572 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 264/504] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:31 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1738 bytes in 552 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:32.021002 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-77bac423-ab78-4c7c-9d80-6f8d6ed9f6e7 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:32.021800 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-77bac423-ab78-4c7c-9d80-6f8d6ed9f6e7 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] [instance: 1c66adb2-72ed-4327-ba4c-efdef9fa18f2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:32.035558 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-77bac423-ab78-4c7c-9d80-6f8d6ed9f6e7 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:32.037124 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-77bac423-ab78-4c7c-9d80-6f8d6ed9f6e7 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:32.040093 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-77bac423-ab78-4c7c-9d80-6f8d6ed9f6e7 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:32.040763 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-77bac423-ab78-4c7c-9d80-6f8d6ed9f6e7 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:32.432402 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-966275e7-3fff-456f-a82d-cf0c00431e00 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:32.442452 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-966275e7-3fff-456f-a82d-cf0c00431e00 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1370 microversion: 2.1 time: 0.579205 Jul 19 11:07:32.461275 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 241/505] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:31 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1370 bytes in 599 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:07:32.528830 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-77bac423-ab78-4c7c-9d80-6f8d6ed9f6e7 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '49e762b40bfd49859d5820d3e45adb1b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b4575dc494d40fda826dbcd0600a76a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:32.555003 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-77bac423-ab78-4c7c-9d80-6f8d6ed9f6e7 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] 10.222.0.33 "GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2" status: 200 len: 1464 microversion: 2.1 time: 0.541497 Jul 19 11:07:32.560474 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 265/506] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:32 2022] GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2 => generated 1464 bytes in 548 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:32.622517 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-f958b61f-426e-4aff-a1ff-ff02bd30e2e2 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:32.623469 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-f958b61f-426e-4aff-a1ff-ff02bd30e2e2 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] [instance: 1c66adb2-72ed-4327-ba4c-efdef9fa18f2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:32.634383 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f958b61f-426e-4aff-a1ff-ff02bd30e2e2 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:32.639482 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f958b61f-426e-4aff-a1ff-ff02bd30e2e2 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:32.640403 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f958b61f-426e-4aff-a1ff-ff02bd30e2e2 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:32.640913 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f958b61f-426e-4aff-a1ff-ff02bd30e2e2 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:32.721344 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-f958b61f-426e-4aff-a1ff-ff02bd30e2e2 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] [instance: 1c66adb2-72ed-4327-ba4c-efdef9fa18f2] Going to try to terminate instance {{(pid=117849) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 19 11:07:32.977113 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-f958b61f-426e-4aff-a1ff-ff02bd30e2e2 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] 10.222.0.33 "DELETE /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2" status: 204 len: 0 microversion: 2.1 time: 0.357011 Jul 19 11:07:32.982896 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 242/507] 10.222.0.33 () {58 vars in 1238 bytes} [Tue Jul 19 11:07:32 2022] DELETE /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 19 11:07:32.992407 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-f87b80bd-ab6e-4385-87fb-3928363f949d tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:32.993353 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-f87b80bd-ab6e-4385-87fb-3928363f949d tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] [instance: 1c66adb2-72ed-4327-ba4c-efdef9fa18f2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:33.002216 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f87b80bd-ab6e-4385-87fb-3928363f949d tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:33.002949 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f87b80bd-ab6e-4385-87fb-3928363f949d tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:33.003948 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f87b80bd-ab6e-4385-87fb-3928363f949d tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:33.005235 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f87b80bd-ab6e-4385-87fb-3928363f949d tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:33.028886 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-8fb91b66-015b-4222-a101-b05fc2e619cb tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:33.030088 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-8fb91b66-015b-4222-a101-b05fc2e619cb tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:33.055437 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8fb91b66-015b-4222-a101-b05fc2e619cb tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:33.057877 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8fb91b66-015b-4222-a101-b05fc2e619cb tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:33.064259 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8fb91b66-015b-4222-a101-b05fc2e619cb tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:33.069288 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8fb91b66-015b-4222-a101-b05fc2e619cb tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:33.380575 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-f87b80bd-ab6e-4385-87fb-3928363f949d tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '49e762b40bfd49859d5820d3e45adb1b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b4575dc494d40fda826dbcd0600a76a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:33.390375 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-f87b80bd-ab6e-4385-87fb-3928363f949d tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] 10.222.0.33 "GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2" status: 200 len: 1470 microversion: 2.1 time: 0.403040 Jul 19 11:07:33.393779 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 266/508] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:32 2022] GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2 => generated 1470 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:33.477989 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-dcb739de-0d83-482d-a6a2-0b4e68ad17f6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:33.478668 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-dcb739de-0d83-482d-a6a2-0b4e68ad17f6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:33.495434 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dcb739de-0d83-482d-a6a2-0b4e68ad17f6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:33.496214 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dcb739de-0d83-482d-a6a2-0b4e68ad17f6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:33.497654 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dcb739de-0d83-482d-a6a2-0b4e68ad17f6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:33.498531 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dcb739de-0d83-482d-a6a2-0b4e68ad17f6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:33.517719 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-8fb91b66-015b-4222-a101-b05fc2e619cb tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:33.529392 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-8fb91b66-015b-4222-a101-b05fc2e619cb tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:33.530905 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-8fb91b66-015b-4222-a101-b05fc2e619cb tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:33.532060 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-8fb91b66-015b-4222-a101-b05fc2e619cb tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1738 microversion: 2.20 time: 0.513694 Jul 19 11:07:33.543344 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 243/509] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:33 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1738 bytes in 515 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:33.798051 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-dcb739de-0d83-482d-a6a2-0b4e68ad17f6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:33.810884 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-dcb739de-0d83-482d-a6a2-0b4e68ad17f6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1370 microversion: 2.1 time: 0.336675 Jul 19 11:07:33.813183 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 267/510] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:33 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1370 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:33.853862 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-5d56d36e-cc8a-4895-9ef4-ef15d715c527 req-d22fc82f-fb1a-4c15-b20a-44b1d928accf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b9897cfb-9ec6-41f4-bcb5-ba1a978421d3", "name": "network-vif-plugged", "status": "completed", "tag": "bee94173-a17a-45b4-ab19-e862338cf16b"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:07:33.870325 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-5d56d36e-cc8a-4895-9ef4-ef15d715c527 req-d22fc82f-fb1a-4c15-b20a-44b1d928accf service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:33.881348 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-5d56d36e-cc8a-4895-9ef4-ef15d715c527 req-d22fc82f-fb1a-4c15-b20a-44b1d928accf service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:33.917637 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-5d56d36e-cc8a-4895-9ef4-ef15d715c527 req-d22fc82f-fb1a-4c15-b20a-44b1d928accf service nova] Creating event network-vif-plugged:bee94173-a17a-45b4-ab19-e862338cf16b for instance b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 on hv-cinder-maste Jul 19 11:07:33.924022 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-5d56d36e-cc8a-4895-9ef4-ef15d715c527 req-d22fc82f-fb1a-4c15-b20a-44b1d928accf service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.075017 Jul 19 11:07:33.925609 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 244/511] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:07:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:07:34.423338 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-26d1b6ae-4b78-46b5-ab5e-cb0db8bc61bc tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:34.425704 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-26d1b6ae-4b78-46b5-ab5e-cb0db8bc61bc tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] [instance: 1c66adb2-72ed-4327-ba4c-efdef9fa18f2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:34.451859 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-26d1b6ae-4b78-46b5-ab5e-cb0db8bc61bc tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:34.454006 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-26d1b6ae-4b78-46b5-ab5e-cb0db8bc61bc tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:34.455215 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-26d1b6ae-4b78-46b5-ab5e-cb0db8bc61bc tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:34.455793 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-26d1b6ae-4b78-46b5-ab5e-cb0db8bc61bc tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:34.553526 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-05ecfbc1-c26b-4577-a161-3944d93ea24e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:34.556313 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-05ecfbc1-c26b-4577-a161-3944d93ea24e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:34.569625 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-05ecfbc1-c26b-4577-a161-3944d93ea24e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:34.576720 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-05ecfbc1-c26b-4577-a161-3944d93ea24e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:34.577753 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-05ecfbc1-c26b-4577-a161-3944d93ea24e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:34.581241 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-05ecfbc1-c26b-4577-a161-3944d93ea24e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:34.651844 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-26d1b6ae-4b78-46b5-ab5e-cb0db8bc61bc tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '49e762b40bfd49859d5820d3e45adb1b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b4575dc494d40fda826dbcd0600a76a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:34.660427 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-26d1b6ae-4b78-46b5-ab5e-cb0db8bc61bc tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] 10.222.0.33 "GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2" status: 200 len: 1470 microversion: 2.1 time: 0.254053 Jul 19 11:07:34.662438 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 268/512] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:34 2022] GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2 => generated 1470 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:34.835567 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-27b9766e-b5c0-4e28-b4db-c6762ba946a3 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:34.836399 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-27b9766e-b5c0-4e28-b4db-c6762ba946a3 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:34.852026 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-27b9766e-b5c0-4e28-b4db-c6762ba946a3 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:34.853066 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-27b9766e-b5c0-4e28-b4db-c6762ba946a3 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:34.856259 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-27b9766e-b5c0-4e28-b4db-c6762ba946a3 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:34.857004 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-27b9766e-b5c0-4e28-b4db-c6762ba946a3 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:34.872273 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-05ecfbc1-c26b-4577-a161-3944d93ea24e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:34.885886 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-05ecfbc1-c26b-4577-a161-3944d93ea24e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:34.892920 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-05ecfbc1-c26b-4577-a161-3944d93ea24e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:34.899646 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-05ecfbc1-c26b-4577-a161-3944d93ea24e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1738 microversion: 2.20 time: 0.347701 Jul 19 11:07:34.901582 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 245/513] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:34 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1738 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:34.931764 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-a0a3e279-7f6d-4cb7-b212-3d2921321661 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:34.938791 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-a0a3e279-7f6d-4cb7-b212-3d2921321661 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:34.965642 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a0a3e279-7f6d-4cb7-b212-3d2921321661 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:34.966391 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a0a3e279-7f6d-4cb7-b212-3d2921321661 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:34.969627 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a0a3e279-7f6d-4cb7-b212-3d2921321661 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:34.970075 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a0a3e279-7f6d-4cb7-b212-3d2921321661 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:35.013151 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-a0a3e279-7f6d-4cb7-b212-3d2921321661 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487/ips" status: 200 len: 145 microversion: 2.1 time: 0.109562 Jul 19 11:07:35.016641 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 246/514] 10.222.0.33 () {58 vars in 1247 bytes} [Tue Jul 19 11:07:34 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487/ips => generated 145 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:07:35.215425 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-27b9766e-b5c0-4e28-b4db-c6762ba946a3 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:35.225559 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-27b9766e-b5c0-4e28-b4db-c6762ba946a3 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1370 microversion: 2.1 time: 0.400052 Jul 19 11:07:35.227215 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 269/515] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:34 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1370 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:35.683184 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-ac535fdd-a404-47d6-ab86-4841187498c0 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:35.684537 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-ac535fdd-a404-47d6-ab86-4841187498c0 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] [instance: 1c66adb2-72ed-4327-ba4c-efdef9fa18f2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:35.710079 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-32213e66-72be-43ba-9650-d094fab8b72c req-5189d0aa-1f08-409b-a71f-fdb68230cd45 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1c66adb2-72ed-4327-ba4c-efdef9fa18f2", "name": "network-vif-deleted", "tag": "eca8d901-e37b-472e-b418-629ae81f029b"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:07:35.711810 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ac535fdd-a404-47d6-ab86-4841187498c0 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:35.717246 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ac535fdd-a404-47d6-ab86-4841187498c0 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:35.718508 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ac535fdd-a404-47d6-ab86-4841187498c0 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:35.719186 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ac535fdd-a404-47d6-ab86-4841187498c0 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:35.732006 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-32213e66-72be-43ba-9650-d094fab8b72c req-5189d0aa-1f08-409b-a71f-fdb68230cd45 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:35.732468 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-32213e66-72be-43ba-9650-d094fab8b72c req-5189d0aa-1f08-409b-a71f-fdb68230cd45 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:35.769015 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-32213e66-72be-43ba-9650-d094fab8b72c req-5189d0aa-1f08-409b-a71f-fdb68230cd45 service nova] Creating event network-vif-deleted:eca8d901-e37b-472e-b418-629ae81f029b for instance 1c66adb2-72ed-4327-ba4c-efdef9fa18f2 on hv-cinder-maste Jul 19 11:07:35.782733 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-32213e66-72be-43ba-9650-d094fab8b72c req-5189d0aa-1f08-409b-a71f-fdb68230cd45 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.076429 Jul 19 11:07:35.788497 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 270/516] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:07:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:07:35.892953 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-ac535fdd-a404-47d6-ab86-4841187498c0 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '49e762b40bfd49859d5820d3e45adb1b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b4575dc494d40fda826dbcd0600a76a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:35.905682 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-ac535fdd-a404-47d6-ab86-4841187498c0 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] 10.222.0.33 "GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2" status: 200 len: 1428 microversion: 2.1 time: 0.230029 Jul 19 11:07:35.906468 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 247/517] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:35 2022] GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2 => generated 1428 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:35.937043 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-292dea71-9859-423a-a302-0bbd85f5f9ee tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:35.937776 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-292dea71-9859-423a-a302-0bbd85f5f9ee tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:35.952764 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-292dea71-9859-423a-a302-0bbd85f5f9ee tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:35.953217 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-292dea71-9859-423a-a302-0bbd85f5f9ee tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:35.954002 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-292dea71-9859-423a-a302-0bbd85f5f9ee tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:35.954443 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-292dea71-9859-423a-a302-0bbd85f5f9ee tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:36.215946 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-292dea71-9859-423a-a302-0bbd85f5f9ee tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:36.239608 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-292dea71-9859-423a-a302-0bbd85f5f9ee tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:36.243485 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-5ae84b6b-f927-4acf-b922-26a38f41a2d6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:36.244115 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-5ae84b6b-f927-4acf-b922-26a38f41a2d6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:36.246603 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-292dea71-9859-423a-a302-0bbd85f5f9ee tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:36.250664 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-292dea71-9859-423a-a302-0bbd85f5f9ee tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1738 microversion: 2.20 time: 0.333363 Jul 19 11:07:36.261811 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 271/518] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:35 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1738 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:36.262026 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5ae84b6b-f927-4acf-b922-26a38f41a2d6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:36.262026 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5ae84b6b-f927-4acf-b922-26a38f41a2d6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:36.262026 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5ae84b6b-f927-4acf-b922-26a38f41a2d6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:36.262026 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5ae84b6b-f927-4acf-b922-26a38f41a2d6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:36.538396 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-5ae84b6b-f927-4acf-b922-26a38f41a2d6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:36.553270 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-5ae84b6b-f927-4acf-b922-26a38f41a2d6 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1370 microversion: 2.1 time: 0.314263 Jul 19 11:07:36.566043 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 248/519] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:36 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1370 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:36.926956 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-1b4f259b-6035-41e4-9320-cf0039117fc6 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:36.928404 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-1b4f259b-6035-41e4-9320-cf0039117fc6 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] [instance: 1c66adb2-72ed-4327-ba4c-efdef9fa18f2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:36.947588 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1b4f259b-6035-41e4-9320-cf0039117fc6 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:36.953289 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1b4f259b-6035-41e4-9320-cf0039117fc6 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:36.954323 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1b4f259b-6035-41e4-9320-cf0039117fc6 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:36.954959 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1b4f259b-6035-41e4-9320-cf0039117fc6 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:37.042287 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1b4f259b-6035-41e4-9320-cf0039117fc6 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:37.052740 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1b4f259b-6035-41e4-9320-cf0039117fc6 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:37.053456 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.objects.instance [None req-1b4f259b-6035-41e4-9320-cf0039117fc6 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lazy-loading 'fault' on Instance uuid 1c66adb2-72ed-4327-ba4c-efdef9fa18f2 {{(pid=117848) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 19 11:07:37.125119 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-1b4f259b-6035-41e4-9320-cf0039117fc6 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '49e762b40bfd49859d5820d3e45adb1b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b4575dc494d40fda826dbcd0600a76a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:37.142381 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-1b4f259b-6035-41e4-9320-cf0039117fc6 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] 10.222.0.33 "GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2" status: 200 len: 1304 microversion: 2.1 time: 0.219528 Jul 19 11:07:37.143655 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 272/520] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:36 2022] GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2 => generated 1304 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:37.269461 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-ce864cd8-43a3-440d-8435-606fb41bbbfc tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:37.270670 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-ce864cd8-43a3-440d-8435-606fb41bbbfc tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:37.291775 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ce864cd8-43a3-440d-8435-606fb41bbbfc tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:37.297782 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ce864cd8-43a3-440d-8435-606fb41bbbfc tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:37.298638 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ce864cd8-43a3-440d-8435-606fb41bbbfc tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:37.299082 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ce864cd8-43a3-440d-8435-606fb41bbbfc tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:37.572032 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-df9e882f-751b-439b-9500-b8b65ef8ce6f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:37.572471 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-df9e882f-751b-439b-9500-b8b65ef8ce6f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:37.581087 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:07:37.589458 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-df9e882f-751b-439b-9500-b8b65ef8ce6f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:37.601716 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-df9e882f-751b-439b-9500-b8b65ef8ce6f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:37.609877 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-df9e882f-751b-439b-9500-b8b65ef8ce6f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:37.611532 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-df9e882f-751b-439b-9500-b8b65ef8ce6f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:37.831581 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-ce864cd8-43a3-440d-8435-606fb41bbbfc tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:37.865726 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-ce864cd8-43a3-440d-8435-606fb41bbbfc tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:37.878453 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-ce864cd8-43a3-440d-8435-606fb41bbbfc tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:37.879694 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-ce864cd8-43a3-440d-8435-606fb41bbbfc tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1738 microversion: 2.20 time: 0.614088 Jul 19 11:07:37.896319 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 249/521] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:37 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1738 bytes in 630 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:38.167154 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-df9e882f-751b-439b-9500-b8b65ef8ce6f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:38.176013 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-613fa6f3-b31d-458b-868a-9757735bdf47 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:38.176252 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-df9e882f-751b-439b-9500-b8b65ef8ce6f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1564 microversion: 2.1 time: 0.608378 Jul 19 11:07:38.187444 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 273/522] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:37 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1564 bytes in 620 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:38.188672 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-613fa6f3-b31d-458b-868a-9757735bdf47 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] [instance: 1c66adb2-72ed-4327-ba4c-efdef9fa18f2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:38.205962 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-613fa6f3-b31d-458b-868a-9757735bdf47 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:38.214044 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-613fa6f3-b31d-458b-868a-9757735bdf47 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:38.218316 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-8541800a-3e01-49fe-95ac-d2fd084dd180 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:38.218972 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-8541800a-3e01-49fe-95ac-d2fd084dd180 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:38.219294 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-613fa6f3-b31d-458b-868a-9757735bdf47 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:38.219995 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-613fa6f3-b31d-458b-868a-9757735bdf47 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:38.230646 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8541800a-3e01-49fe-95ac-d2fd084dd180 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:38.240046 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8541800a-3e01-49fe-95ac-d2fd084dd180 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:38.241310 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8541800a-3e01-49fe-95ac-d2fd084dd180 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:38.242396 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8541800a-3e01-49fe-95ac-d2fd084dd180 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:38.276277 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.wsgi [None req-613fa6f3-b31d-458b-868a-9757735bdf47 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] HTTP exception thrown: Instance 1c66adb2-72ed-4327-ba4c-efdef9fa18f2 could not be found. Jul 19 11:07:38.278786 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-613fa6f3-b31d-458b-868a-9757735bdf47 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Request method failure captured: Jul 19 11:07:38.278786 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: request: GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2 HTTP/1.1 Jul 19 11:07:38.278786 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept: application/json Jul 19 11:07:38.278786 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept-Encoding: identity Jul 19 11:07:38.278786 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Connection: close Jul 19 11:07:38.278786 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Length: 0 Jul 19 11:07:38.278786 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Type: application/json Jul 19 11:07:38.278786 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Host: 10.222.0.33 Jul 19 11:07:38.278786 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Openstack-System-Scope: None Jul 19 11:07:38.278786 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: User-Agent: python-urllib3/1.26.9 Jul 19 11:07:38.278786 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Auth-Token: gAAAAABi1pBcTsK5ANz4o5Ybl9_Kfjn-Fg5xt3mmNf5zUrKhBcxVWK3X83E988JCYG7TGvnrEvfelnV1Q5dJbOc9GQeGvxBO0lCEp824yfsu1tYscMiUAOrihxioM_0Bd86pHSKul1GEnAvopGqvby6UifHTUzJ5GsjaQg3-w6rxkw8JK7orhEc Jul 19 11:07:38.278786 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Id: None Jul 19 11:07:38.278786 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Name: None Jul 19 11:07:38.278786 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Identity-Status: Confirmed Jul 19 11:07:38.278786 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Is-Admin-Project: True Jul 19 11:07:38.278786 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Id: default Jul 19 11:07:38.278786 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Name: Default Jul 19 11:07:38.278786 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Id: 2b4575dc494d40fda826dbcd0600a76a Jul 19 11:07:38.278786 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Name: tempest-VolumesNegativeTest-1175169626 Jul 19 11:07:38.278786 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Role: member,reader Jul 19 11:07:38.278786 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Roles: member,reader Jul 19 11:07:38.280404 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/2b4575dc494d40fda826dbcd0600a76a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/2b4575dc494d40fda826dbcd0600a76a"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/2b4575dc494d40fda826dbcd0600a76a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_2b4575dc494d40fda826dbcd0600a76a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:07:38.280404 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant: tempest-VolumesNegativeTest-1175169626 Jul 19 11:07:38.280404 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Id: 2b4575dc494d40fda826dbcd0600a76a Jul 19 11:07:38.280404 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Name: tempest-VolumesNegativeTest-1175169626 Jul 19 11:07:38.280404 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User: tempest-VolumesNegativeTest-1175169626-project Jul 19 11:07:38.280404 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Id: default Jul 19 11:07:38.280404 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Name: Default Jul 19 11:07:38.280404 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Id: 49e762b40bfd49859d5820d3e45adb1b Jul 19 11:07:38.280404 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Name: tempest-VolumesNegativeTest-1175169626-project Jul 19 11:07:38.280404 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: method: > Jul 19 11:07:38.280404 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: exception: Instance 1c66adb2-72ed-4327-ba4c-efdef9fa18f2 could not be found. Jul 19 11:07:38.280404 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: action_args: {'id': '1c66adb2-72ed-4327-ba4c-efdef9fa18f2'} Jul 19 11:07:38.281449 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:07:38.281449 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:07:38.281449 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:07:38.281449 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:07:38.281449 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:07:38.281449 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:07:38.281449 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:07:38.281449 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:07:38.281449 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:07:38.281449 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:07:38.281449 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1c66adb2-72ed-4327-ba4c-efdef9fa18f2 could not be found. Jul 19 11:07:38.281449 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:07:38.281449 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:07:38.281449 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:07:38.281449 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:07:38.281449 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:07:38.281449 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:07:38.281449 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:07:38.281449 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:07:38.281449 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:07:38.282887 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:07:38.282887 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 19 11:07:38.282887 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 19 11:07:38.282887 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:07:38.282887 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:07:38.282887 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:07:38.282887 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:07:38.282887 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1c66adb2-72ed-4327-ba4c-efdef9fa18f2 could not be found. Jul 19 11:07:38.282887 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:07:38.282887 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:07:38.282887 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:07:38.282887 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:07:38.282887 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:07:38.282887 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:07:38.282887 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:07:38.282887 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:07:38.282887 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1c66adb2-72ed-4327-ba4c-efdef9fa18f2 could not be found. Jul 19 11:07:38.282887 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:07:38.285283 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-613fa6f3-b31d-458b-868a-9757735bdf47 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] Returning 404 to user: Instance 1c66adb2-72ed-4327-ba4c-efdef9fa18f2 could not be found. {{(pid=117849) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:07:38.291341 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-613fa6f3-b31d-458b-868a-9757735bdf47 tempest-VolumesNegativeTest-1175169626 tempest-VolumesNegativeTest-1175169626-project] 10.222.0.33 "GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2" status: 404 len: 111 microversion: 2.1 time: 0.135655 Jul 19 11:07:38.292568 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 250/523] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:38 2022] GET /compute/v2.1/servers/1c66adb2-72ed-4327-ba4c-efdef9fa18f2 => generated 111 bytes in 137 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 19 11:07:38.431407 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-8541800a-3e01-49fe-95ac-d2fd084dd180 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.238855 Jul 19 11:07:38.438326 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 274/524] 10.222.0.33 () {58 vars in 1274 bytes} [Tue Jul 19 11:07:38 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3/os-interface => generated 284 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:07:38.899379 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-6b885365-4841-4dac-bca9-5a388665213d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:38.899901 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-6b885365-4841-4dac-bca9-5a388665213d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:38.910192 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6b885365-4841-4dac-bca9-5a388665213d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:38.910640 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6b885365-4841-4dac-bca9-5a388665213d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:38.911433 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6b885365-4841-4dac-bca9-5a388665213d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:38.911882 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6b885365-4841-4dac-bca9-5a388665213d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:39.525592 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-6b885365-4841-4dac-bca9-5a388665213d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:39.546413 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-6b885365-4841-4dac-bca9-5a388665213d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:39.548649 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-6b885365-4841-4dac-bca9-5a388665213d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:39.549896 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-6b885365-4841-4dac-bca9-5a388665213d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1738 microversion: 2.20 time: 0.654864 Jul 19 11:07:39.553014 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 251/525] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:38 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1738 bytes in 658 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:40.409671 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-712dfd6b-d3b5-4b87-9a2d-f18ff4872947 req-9f16446b-fbd2-4c52-a9d8-cfcbf1cb0762 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b9897cfb-9ec6-41f4-bcb5-ba1a978421d3", "tag": "bee94173-a17a-45b4-ab19-e862338cf16b"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:07:40.418009 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-712dfd6b-d3b5-4b87-9a2d-f18ff4872947 req-9f16446b-fbd2-4c52-a9d8-cfcbf1cb0762 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:40.418742 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-712dfd6b-d3b5-4b87-9a2d-f18ff4872947 req-9f16446b-fbd2-4c52-a9d8-cfcbf1cb0762 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:40.432359 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-712dfd6b-d3b5-4b87-9a2d-f18ff4872947 req-9f16446b-fbd2-4c52-a9d8-cfcbf1cb0762 service nova] Creating event network-changed:bee94173-a17a-45b4-ab19-e862338cf16b for instance b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 on hv-cinder-maste Jul 19 11:07:40.440197 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-712dfd6b-d3b5-4b87-9a2d-f18ff4872947 req-9f16446b-fbd2-4c52-a9d8-cfcbf1cb0762 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032908 Jul 19 11:07:40.441563 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 275/526] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:07:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:07:40.568367 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-c7b6ae76-09d7-4c67-90cf-e0daf08f7695 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:40.569344 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-c7b6ae76-09d7-4c67-90cf-e0daf08f7695 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:40.591047 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c7b6ae76-09d7-4c67-90cf-e0daf08f7695 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:40.594729 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c7b6ae76-09d7-4c67-90cf-e0daf08f7695 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:40.597699 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c7b6ae76-09d7-4c67-90cf-e0daf08f7695 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:40.601261 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c7b6ae76-09d7-4c67-90cf-e0daf08f7695 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:40.779086 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-7b903a05-b7a8-4237-9f9c-a84899e25e36 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "1a8a01af-e70f-42d8-b1ae-24a6cd72175d"}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:07:40.781268 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-7b903a05-b7a8-4237-9f9c-a84899e25e36 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:40.810248 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7b903a05-b7a8-4237-9f9c-a84899e25e36 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:40.813377 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7b903a05-b7a8-4237-9f9c-a84899e25e36 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:40.818102 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7b903a05-b7a8-4237-9f9c-a84899e25e36 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:40.820264 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7b903a05-b7a8-4237-9f9c-a84899e25e36 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:40.868936 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-c7b6ae76-09d7-4c67-90cf-e0daf08f7695 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:40.877756 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-c7b6ae76-09d7-4c67-90cf-e0daf08f7695 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:40.881903 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-c7b6ae76-09d7-4c67-90cf-e0daf08f7695 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:40.889660 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-c7b6ae76-09d7-4c67-90cf-e0daf08f7695 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1738 microversion: 2.20 time: 0.325907 Jul 19 11:07:40.897488 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 252/527] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:40 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1738 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:40.964517 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=100 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:07:41.318333 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-7b903a05-b7a8-4237-9f9c-a84899e25e36 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "POST /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.545807 Jul 19 11:07:41.319542 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 276/528] 10.222.0.33 () {60 vars in 1322 bytes} [Tue Jul 19 11:07:40 2022] POST /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487/os-volume_attachments => generated 194 bytes in 550 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:07:41.923139 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-99a2ac28-7e01-455b-becd-f6da04303bd0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:41.923742 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-99a2ac28-7e01-455b-becd-f6da04303bd0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:41.935920 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-99a2ac28-7e01-455b-becd-f6da04303bd0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:41.936380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-99a2ac28-7e01-455b-becd-f6da04303bd0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:41.937204 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-99a2ac28-7e01-455b-becd-f6da04303bd0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:41.937645 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-99a2ac28-7e01-455b-becd-f6da04303bd0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:41.999301 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:07:42.162504 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-99a2ac28-7e01-455b-becd-f6da04303bd0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:42.181315 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-99a2ac28-7e01-455b-becd-f6da04303bd0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:42.182993 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-99a2ac28-7e01-455b-becd-f6da04303bd0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:42.185763 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-99a2ac28-7e01-455b-becd-f6da04303bd0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1738 microversion: 2.20 time: 0.273283 Jul 19 11:07:42.187048 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 253/529] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:41 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1738 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:07:43.206528 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-73d8a975-8770-4b55-89aa-3ee28a693db6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:43.208150 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-73d8a975-8770-4b55-89aa-3ee28a693db6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:43.215604 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-73d8a975-8770-4b55-89aa-3ee28a693db6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:43.216310 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-73d8a975-8770-4b55-89aa-3ee28a693db6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:43.217438 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-73d8a975-8770-4b55-89aa-3ee28a693db6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:43.217974 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-73d8a975-8770-4b55-89aa-3ee28a693db6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:43.478946 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-73d8a975-8770-4b55-89aa-3ee28a693db6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:43.489390 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-73d8a975-8770-4b55-89aa-3ee28a693db6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:43.490914 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-73d8a975-8770-4b55-89aa-3ee28a693db6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:43.492157 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-73d8a975-8770-4b55-89aa-3ee28a693db6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1720 microversion: 2.20 time: 0.291519 Jul 19 11:07:43.492799 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 277/530] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:43 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1720 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:44.511901 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-f85edc94-e1aa-4887-85aa-222bd1f2a496 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:44.512513 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-f85edc94-e1aa-4887-85aa-222bd1f2a496 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:44.523736 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f85edc94-e1aa-4887-85aa-222bd1f2a496 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:44.524411 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f85edc94-e1aa-4887-85aa-222bd1f2a496 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:44.525508 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f85edc94-e1aa-4887-85aa-222bd1f2a496 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:44.526098 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f85edc94-e1aa-4887-85aa-222bd1f2a496 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:44.691948 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-f85edc94-e1aa-4887-85aa-222bd1f2a496 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:44.700862 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-f85edc94-e1aa-4887-85aa-222bd1f2a496 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:44.702689 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-f85edc94-e1aa-4887-85aa-222bd1f2a496 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:44.704046 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-f85edc94-e1aa-4887-85aa-222bd1f2a496 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1720 microversion: 2.20 time: 0.196509 Jul 19 11:07:44.704891 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 254/531] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:44 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1720 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:45.727214 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-eb4249a1-cc50-4ecd-bcfc-8b97207aca96 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:45.727214 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-eb4249a1-cc50-4ecd-bcfc-8b97207aca96 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:45.739362 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-eb4249a1-cc50-4ecd-bcfc-8b97207aca96 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:45.741572 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-eb4249a1-cc50-4ecd-bcfc-8b97207aca96 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:45.742448 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-eb4249a1-cc50-4ecd-bcfc-8b97207aca96 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:45.744898 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-eb4249a1-cc50-4ecd-bcfc-8b97207aca96 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:45.910051 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-eb4249a1-cc50-4ecd-bcfc-8b97207aca96 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:45.919903 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-eb4249a1-cc50-4ecd-bcfc-8b97207aca96 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:45.921449 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-eb4249a1-cc50-4ecd-bcfc-8b97207aca96 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:45.925218 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-eb4249a1-cc50-4ecd-bcfc-8b97207aca96 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1720 microversion: 2.20 time: 0.204724 Jul 19 11:07:45.926683 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 278/532] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:45 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1720 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:46.944647 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-bdcc2f5c-f2b0-437b-baa2-50ae752f8cef tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:46.945395 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-bdcc2f5c-f2b0-437b-baa2-50ae752f8cef tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:46.951600 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bdcc2f5c-f2b0-437b-baa2-50ae752f8cef tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:46.952254 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bdcc2f5c-f2b0-437b-baa2-50ae752f8cef tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:46.953196 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bdcc2f5c-f2b0-437b-baa2-50ae752f8cef tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:46.953798 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bdcc2f5c-f2b0-437b-baa2-50ae752f8cef tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:47.146168 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-bdcc2f5c-f2b0-437b-baa2-50ae752f8cef tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:47.158150 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-bdcc2f5c-f2b0-437b-baa2-50ae752f8cef tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:47.161916 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-bdcc2f5c-f2b0-437b-baa2-50ae752f8cef tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:47.163551 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-bdcc2f5c-f2b0-437b-baa2-50ae752f8cef tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1663 microversion: 2.20 time: 0.224113 Jul 19 11:07:47.164322 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 255/533] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:46 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1663 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:47.186887 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-104ffa38-6a5b-4b55-b114-13b65e450234 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "66afeab5-b4b2-42fc-8273-20e9cfb6bbe8"}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:07:47.188106 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-104ffa38-6a5b-4b55-b114-13b65e450234 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:47.200599 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-104ffa38-6a5b-4b55-b114-13b65e450234 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:47.201345 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-104ffa38-6a5b-4b55-b114-13b65e450234 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:47.203479 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-104ffa38-6a5b-4b55-b114-13b65e450234 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:47.204242 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-104ffa38-6a5b-4b55-b114-13b65e450234 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:47.338443 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-868bd537-df5a-44cb-9021-7e255a3df139 req-86ba5237-d8de-42cf-81c2-fa3cd150d774 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f52fbfae-577b-403d-a4d3-9a4b79ae2326", "name": "network-vif-unplugged", "status": "completed", "tag": "748a7315-79e9-4ab8-ad6a-336c057e8b2c"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:07:47.354737 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_api:SELECT=76,nova_api:UPDATE=1 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:07:47.357364 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-868bd537-df5a-44cb-9021-7e255a3df139 req-86ba5237-d8de-42cf-81c2-fa3cd150d774 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:47.362355 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-868bd537-df5a-44cb-9021-7e255a3df139 req-86ba5237-d8de-42cf-81c2-fa3cd150d774 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:47.378523 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.compute.server_external_events [req-868bd537-df5a-44cb-9021-7e255a3df139 req-86ba5237-d8de-42cf-81c2-fa3cd150d774 service nova] Unable to find a host for instance f52fbfae-577b-403d-a4d3-9a4b79ae2326. Dropping event network-vif-unplugged {{(pid=117849) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:143}} Jul 19 11:07:47.379664 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-868bd537-df5a-44cb-9021-7e255a3df139 req-86ba5237-d8de-42cf-81c2-fa3cd150d774 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.045063 Jul 19 11:07:47.380400 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 256/534] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:07:47 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 46 msecs (HTTP/1.1 207) 9 headers in 367 bytes (2 switches on core 0) Jul 19 11:07:47.567740 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-1f7c7bce-f860-4be7-9c5f-8e85e0f9474f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Action: 'create', calling method: .version_select at 0x7f04aa5ae280>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-468494935", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLDGUySLFozgbAo4OmYvn8ejPpBLg7Hpr5Pi/2evP0EHUICI7T2kOJPjr2TRLcS8E39hMi1JCV2S8vp1v+7e2sISKvanSkTaV6XqaOiGCCbpEEWkCJf7E/HIPzsbc4EHDg=="}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:07:47.583217 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-1f7c7bce-f860-4be7-9c5f-8e85e0f9474f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Getting quotas for project 05630eeaf84a4d1bafd213d419f7a97a. Resources: {'key_pairs'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:07:47.587388 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-1f7c7bce-f860-4be7-9c5f-8e85e0f9474f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Getting quotas for user 9e853b49c2c744e89243ec51e7a1a379 and project 05630eeaf84a4d1bafd213d419f7a97a. Resources: {'key_pairs'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:07:47.614729 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-1f7c7bce-f860-4be7-9c5f-8e85e0f9474f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "POST /compute/v2.1/os-keypairs" status: 200 len: 402 microversion: 2.1 time: 0.051916 Jul 19 11:07:47.615307 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-104ffa38-6a5b-4b55-b114-13b65e450234 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "POST /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.437211 Jul 19 11:07:47.615458 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 257/535] 10.222.0.33 () {60 vars in 1158 bytes} [Tue Jul 19 11:07:47 2022] POST /compute/v2.1/os-keypairs => generated 402 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:07:47.617608 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 279/536] 10.222.0.33 () {62 vars in 1363 bytes} [Tue Jul 19 11:07:47 2022] POST /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326/os-volume_attachments => generated 188 bytes in 440 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 19 11:07:47.736036 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-faf1ccf9-c50c-46e8-a7f2-e9bd20e05bdc tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-1064785736", "imageRef": "77b576ff-73ad-4857-9e87-1b396373b057", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-468494935", "networks": [{"uuid": "c45e8b75-1fcc-418c-baa1-a3821825a45d"}]}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:07:47.747574 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-7713a8e2-30c8-4022-a2b8-49b9d7292277 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:07:47.748627 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-7713a8e2-30c8-4022-a2b8-49b9d7292277 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:47.756114 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7713a8e2-30c8-4022-a2b8-49b9d7292277 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:47.756336 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7713a8e2-30c8-4022-a2b8-49b9d7292277 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:47.757169 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7713a8e2-30c8-4022-a2b8-49b9d7292277 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:47.758328 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7713a8e2-30c8-4022-a2b8-49b9d7292277 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:47.796979 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.network.neutron [None req-faf1ccf9-c50c-46e8-a7f2-e9bd20e05bdc tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] validate_networks() for [('c45e8b75-1fcc-418c-baa1-a3821825a45d', None, None, None, None, None)] {{(pid=117849) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 19 11:07:47.863635 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-7713a8e2-30c8-4022-a2b8-49b9d7292277 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "POST /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326/action" status: 202 len: 0 microversion: 2.20 time: 0.121504 Jul 19 11:07:47.864543 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 280/537] 10.222.0.33 () {62 vars in 1318 bytes} [Tue Jul 19 11:07:47 2022] POST /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326/action => generated 0 bytes in 123 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 19 11:07:47.883046 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-24c6a9af-a8eb-4b28-813e-eb4b619e2c7d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:47.883765 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-24c6a9af-a8eb-4b28-813e-eb4b619e2c7d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:47.891164 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-24c6a9af-a8eb-4b28-813e-eb4b619e2c7d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:47.891794 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-24c6a9af-a8eb-4b28-813e-eb4b619e2c7d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:47.893348 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-24c6a9af-a8eb-4b28-813e-eb4b619e2c7d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:47.895339 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-24c6a9af-a8eb-4b28-813e-eb4b619e2c7d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:48.084431 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-24c6a9af-a8eb-4b28-813e-eb4b619e2c7d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:48.094841 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-24c6a9af-a8eb-4b28-813e-eb4b619e2c7d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:48.097008 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-24c6a9af-a8eb-4b28-813e-eb4b619e2c7d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:48.098172 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-24c6a9af-a8eb-4b28-813e-eb4b619e2c7d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1749 microversion: 2.20 time: 0.229154 Jul 19 11:07:48.098894 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 281/538] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:47 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1749 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:07:48.293902 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-faf1ccf9-c50c-46e8-a7f2-e9bd20e05bdc tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Flavor limits 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 19 11:07:48.294470 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-faf1ccf9-c50c-46e8-a7f2-e9bd20e05bdc tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Image limits 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 19 11:07:48.295017 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-faf1ccf9-c50c-46e8-a7f2-e9bd20e05bdc tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Flavor pref 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 19 11:07:48.295419 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-faf1ccf9-c50c-46e8-a7f2-e9bd20e05bdc tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Image pref 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 19 11:07:48.296017 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-faf1ccf9-c50c-46e8-a7f2-e9bd20e05bdc tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 19 11:07:48.306698 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-faf1ccf9-c50c-46e8-a7f2-e9bd20e05bdc tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:48.307142 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-faf1ccf9-c50c-46e8-a7f2-e9bd20e05bdc tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:48.308543 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-faf1ccf9-c50c-46e8-a7f2-e9bd20e05bdc tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:48.309178 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-faf1ccf9-c50c-46e8-a7f2-e9bd20e05bdc tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:48.327161 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-faf1ccf9-c50c-46e8-a7f2-e9bd20e05bdc tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Getting quotas for project 05630eeaf84a4d1bafd213d419f7a97a. Resources: {'cores', 'instances', 'ram'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:07:48.331494 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-faf1ccf9-c50c-46e8-a7f2-e9bd20e05bdc tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Getting quotas for user 9e853b49c2c744e89243ec51e7a1a379 and project 05630eeaf84a4d1bafd213d419f7a97a. Resources: {'cores', 'instances', 'ram'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:07:48.343447 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-faf1ccf9-c50c-46e8-a7f2-e9bd20e05bdc tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Going to run 1 instances... {{(pid=117849) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 19 11:07:48.345848 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-faf1ccf9-c50c-46e8-a7f2-e9bd20e05bdc tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='77b576ff-73ad-4857-9e87-1b396373b057',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117849) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 19 11:07:48.396682 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-faf1ccf9-c50c-46e8-a7f2-e9bd20e05bdc tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.664455 Jul 19 11:07:48.398700 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 258/539] 10.222.0.33 () {60 vars in 1146 bytes} [Tue Jul 19 11:07:47 2022] POST /compute/v2.1/servers => generated 380 bytes in 667 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 19 11:07:48.414292 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-aeb80a49-ea46-45b3-b1ca-b57208a934d7 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:48.416215 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-aeb80a49-ea46-45b3-b1ca-b57208a934d7 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:48.559159 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-aeb80a49-ea46-45b3-b1ca-b57208a934d7 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:48.568558 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-aeb80a49-ea46-45b3-b1ca-b57208a934d7 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1258 microversion: 2.1 time: 0.159435 Jul 19 11:07:48.574391 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 282/540] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:48 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1258 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:49.081677 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-1a56adf0-b5dd-427c-86c8-304f0c212e3f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:07:49.089103 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1a56adf0-b5dd-427c-86c8-304f0c212e3f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:49.089839 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1a56adf0-b5dd-427c-86c8-304f0c212e3f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:49.117931 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-5e516ebf-cda3-4188-be48-9a354370b055 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:49.118726 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-5e516ebf-cda3-4188-be48-9a354370b055 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:49.127538 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5e516ebf-cda3-4188-be48-9a354370b055 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:49.128455 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5e516ebf-cda3-4188-be48-9a354370b055 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:49.129575 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5e516ebf-cda3-4188-be48-9a354370b055 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:49.130477 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5e516ebf-cda3-4188-be48-9a354370b055 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:49.209865 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-1a56adf0-b5dd-427c-86c8-304f0c212e3f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Going to try to stop instance {{(pid=117849) force_stop /opt/stack/nova/nova/compute/api.py:2707}} Jul 19 11:07:49.263559 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-1a56adf0-b5dd-427c-86c8-304f0c212e3f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "POST /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487/action" status: 202 len: 0 microversion: 2.1 time: 0.185957 Jul 19 11:07:49.264406 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 259/541] 10.222.0.33 () {60 vars in 1277 bytes} [Tue Jul 19 11:07:49 2022] POST /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487/action => generated 0 bytes in 188 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 19 11:07:49.273977 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-19727b5e-6208-4ca8-9958-0e0666db93f5 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:49.275550 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-19727b5e-6208-4ca8-9958-0e0666db93f5 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:49.286042 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-19727b5e-6208-4ca8-9958-0e0666db93f5 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:49.287469 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-19727b5e-6208-4ca8-9958-0e0666db93f5 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:49.289065 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-19727b5e-6208-4ca8-9958-0e0666db93f5 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:49.289673 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-19727b5e-6208-4ca8-9958-0e0666db93f5 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:49.439756 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-19727b5e-6208-4ca8-9958-0e0666db93f5 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:49.464198 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-19727b5e-6208-4ca8-9958-0e0666db93f5 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1723 microversion: 2.1 time: 0.180743 Jul 19 11:07:49.465244 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 260/542] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:49 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1723 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:49.591101 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-820db042-6ca1-473c-91b8-8025ecce70ca tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:49.591710 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-820db042-6ca1-473c-91b8-8025ecce70ca tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:49.598056 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-820db042-6ca1-473c-91b8-8025ecce70ca tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:49.598604 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-820db042-6ca1-473c-91b8-8025ecce70ca tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:49.599635 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-820db042-6ca1-473c-91b8-8025ecce70ca tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:49.600232 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-820db042-6ca1-473c-91b8-8025ecce70ca tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:49.664759 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-820db042-6ca1-473c-91b8-8025ecce70ca tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:49.665393 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-5e516ebf-cda3-4188-be48-9a354370b055 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:49.673849 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-820db042-6ca1-473c-91b8-8025ecce70ca tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1310 microversion: 2.1 time: 0.086378 Jul 19 11:07:49.683849 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 261/543] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:49 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1310 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:07:49.702047 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-5e516ebf-cda3-4188-be48-9a354370b055 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:49.703824 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-5e516ebf-cda3-4188-be48-9a354370b055 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:49.705559 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-5e516ebf-cda3-4188-be48-9a354370b055 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1749 microversion: 2.20 time: 0.592481 Jul 19 11:07:49.706352 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 283/544] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:49 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1749 bytes in 594 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:07:50.482095 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-e52c305f-c271-46bb-8ef9-20a18825af35 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:50.482802 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-e52c305f-c271-46bb-8ef9-20a18825af35 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:50.488892 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e52c305f-c271-46bb-8ef9-20a18825af35 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:50.489415 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e52c305f-c271-46bb-8ef9-20a18825af35 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:50.490265 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e52c305f-c271-46bb-8ef9-20a18825af35 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:50.490760 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e52c305f-c271-46bb-8ef9-20a18825af35 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:50.691046 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-b99cca4f-df0c-442d-90c8-7d0b91cd11f3 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:50.691658 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-b99cca4f-df0c-442d-90c8-7d0b91cd11f3 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:50.704877 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b99cca4f-df0c-442d-90c8-7d0b91cd11f3 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:50.706693 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b99cca4f-df0c-442d-90c8-7d0b91cd11f3 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:50.708596 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b99cca4f-df0c-442d-90c8-7d0b91cd11f3 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:50.709077 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b99cca4f-df0c-442d-90c8-7d0b91cd11f3 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:50.834953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-b99cca4f-df0c-442d-90c8-7d0b91cd11f3 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:50.842994 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-b99cca4f-df0c-442d-90c8-7d0b91cd11f3 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1316 microversion: 2.1 time: 0.155750 Jul 19 11:07:50.845120 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 284/545] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:50 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1316 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:07:50.854685 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-34a00e9b-dc79-4bb8-9313-cda0ef2f3a4a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:50.859336 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-34a00e9b-dc79-4bb8-9313-cda0ef2f3a4a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:50.873728 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-34a00e9b-dc79-4bb8-9313-cda0ef2f3a4a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:50.877199 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-34a00e9b-dc79-4bb8-9313-cda0ef2f3a4a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:50.878229 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-34a00e9b-dc79-4bb8-9313-cda0ef2f3a4a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:50.880731 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-34a00e9b-dc79-4bb8-9313-cda0ef2f3a4a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:50.947617 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-e52c305f-c271-46bb-8ef9-20a18825af35 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:50.974330 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-e52c305f-c271-46bb-8ef9-20a18825af35 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1723 microversion: 2.1 time: 0.494047 Jul 19 11:07:50.975450 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 262/546] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:50 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1723 bytes in 498 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:51.393421 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-34a00e9b-dc79-4bb8-9313-cda0ef2f3a4a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:51.412902 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-34a00e9b-dc79-4bb8-9313-cda0ef2f3a4a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:51.419020 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-34a00e9b-dc79-4bb8-9313-cda0ef2f3a4a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:51.420344 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-34a00e9b-dc79-4bb8-9313-cda0ef2f3a4a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1751 microversion: 2.20 time: 0.572874 Jul 19 11:07:51.434073 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 285/547] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:50 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1751 bytes in 587 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:51.864433 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-940c22db-fcbd-4ade-bc0b-47934f35661b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:51.866004 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-940c22db-fcbd-4ade-bc0b-47934f35661b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:51.877628 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-940c22db-fcbd-4ade-bc0b-47934f35661b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:51.881370 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-940c22db-fcbd-4ade-bc0b-47934f35661b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:51.883012 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-940c22db-fcbd-4ade-bc0b-47934f35661b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:51.893917 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-940c22db-fcbd-4ade-bc0b-47934f35661b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:51.994614 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-00b75f96-0b7f-4a4c-8dbc-e14bb875c2be tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:51.995865 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-00b75f96-0b7f-4a4c-8dbc-e14bb875c2be tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:52.012554 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-00b75f96-0b7f-4a4c-8dbc-e14bb875c2be tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:52.013848 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-00b75f96-0b7f-4a4c-8dbc-e14bb875c2be tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:52.014856 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-00b75f96-0b7f-4a4c-8dbc-e14bb875c2be tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:52.015516 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-00b75f96-0b7f-4a4c-8dbc-e14bb875c2be tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:52.144734 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-940c22db-fcbd-4ade-bc0b-47934f35661b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:52.153084 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-940c22db-fcbd-4ade-bc0b-47934f35661b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1358 microversion: 2.1 time: 0.293846 Jul 19 11:07:52.157673 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 263/548] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:51 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1358 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:52.404773 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-00b75f96-0b7f-4a4c-8dbc-e14bb875c2be tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:52.415840 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-00b75f96-0b7f-4a4c-8dbc-e14bb875c2be tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1723 microversion: 2.1 time: 0.425813 Jul 19 11:07:52.417071 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 286/549] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:51 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1723 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:52.441400 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-0a1188db-1e76-46fe-9ae8-2facb3851ef6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:52.442383 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-0a1188db-1e76-46fe-9ae8-2facb3851ef6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:52.448982 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0a1188db-1e76-46fe-9ae8-2facb3851ef6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:52.449576 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0a1188db-1e76-46fe-9ae8-2facb3851ef6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:52.450452 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0a1188db-1e76-46fe-9ae8-2facb3851ef6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:52.450993 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0a1188db-1e76-46fe-9ae8-2facb3851ef6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:52.601991 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-0a1188db-1e76-46fe-9ae8-2facb3851ef6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:52.612970 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-0a1188db-1e76-46fe-9ae8-2facb3851ef6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:52.614928 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-0a1188db-1e76-46fe-9ae8-2facb3851ef6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:52.616221 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-0a1188db-1e76-46fe-9ae8-2facb3851ef6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1751 microversion: 2.20 time: 0.178734 Jul 19 11:07:52.617582 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 264/550] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:52 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1751 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:53.177103 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-48f35806-88e1-492c-bf8f-683531100d8e tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:53.179810 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-48f35806-88e1-492c-bf8f-683531100d8e tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:53.190992 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-48f35806-88e1-492c-bf8f-683531100d8e tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:53.191452 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-48f35806-88e1-492c-bf8f-683531100d8e tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:53.192238 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-48f35806-88e1-492c-bf8f-683531100d8e tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:53.192682 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-48f35806-88e1-492c-bf8f-683531100d8e tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:53.433556 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-48f35806-88e1-492c-bf8f-683531100d8e tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:53.437093 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-5bd0e50b-19e1-47ff-9237-fda33a74e62c tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:53.438337 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-5bd0e50b-19e1-47ff-9237-fda33a74e62c tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:53.449534 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5bd0e50b-19e1-47ff-9237-fda33a74e62c tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:53.450709 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5bd0e50b-19e1-47ff-9237-fda33a74e62c tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:53.451715 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5bd0e50b-19e1-47ff-9237-fda33a74e62c tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:53.452300 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5bd0e50b-19e1-47ff-9237-fda33a74e62c tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:53.457227 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-48f35806-88e1-492c-bf8f-683531100d8e tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1358 microversion: 2.1 time: 0.285321 Jul 19 11:07:53.457899 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 287/551] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:53 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1358 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:53.635898 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-72b09a3e-f074-4d2f-9ed2-8e24d0152868 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:53.636501 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-72b09a3e-f074-4d2f-9ed2-8e24d0152868 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:53.643554 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-72b09a3e-f074-4d2f-9ed2-8e24d0152868 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:53.648771 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-72b09a3e-f074-4d2f-9ed2-8e24d0152868 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:53.656753 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-72b09a3e-f074-4d2f-9ed2-8e24d0152868 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:53.657445 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-72b09a3e-f074-4d2f-9ed2-8e24d0152868 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:53.675699 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-5bd0e50b-19e1-47ff-9237-fda33a74e62c tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:53.695804 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-5bd0e50b-19e1-47ff-9237-fda33a74e62c tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1723 microversion: 2.1 time: 0.264631 Jul 19 11:07:53.707356 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 265/552] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:53 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1723 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:54.010738 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-72b09a3e-f074-4d2f-9ed2-8e24d0152868 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:54.044629 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-72b09a3e-f074-4d2f-9ed2-8e24d0152868 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:54.050174 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-72b09a3e-f074-4d2f-9ed2-8e24d0152868 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:54.059371 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-72b09a3e-f074-4d2f-9ed2-8e24d0152868 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1751 microversion: 2.20 time: 0.427753 Jul 19 11:07:54.059965 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 288/553] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:53 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1751 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:07:54.475639 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-25501a5e-63a9-4a70-9f92-cf5d92384d2c tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:54.476336 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-25501a5e-63a9-4a70-9f92-cf5d92384d2c tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:54.484307 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-25501a5e-63a9-4a70-9f92-cf5d92384d2c tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:54.484879 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-25501a5e-63a9-4a70-9f92-cf5d92384d2c tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:54.485737 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-25501a5e-63a9-4a70-9f92-cf5d92384d2c tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:54.486178 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-25501a5e-63a9-4a70-9f92-cf5d92384d2c tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:54.662863 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-25501a5e-63a9-4a70-9f92-cf5d92384d2c tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:54.675488 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-25501a5e-63a9-4a70-9f92-cf5d92384d2c tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1358 microversion: 2.1 time: 0.203841 Jul 19 11:07:54.676184 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 266/554] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:54 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1358 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:07:54.715893 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-c8963f42-d594-4be6-a4da-930a8bf3d4cc tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:54.716619 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-c8963f42-d594-4be6-a4da-930a8bf3d4cc tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:54.725876 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c8963f42-d594-4be6-a4da-930a8bf3d4cc tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:54.726639 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c8963f42-d594-4be6-a4da-930a8bf3d4cc tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:54.727617 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c8963f42-d594-4be6-a4da-930a8bf3d4cc tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:54.728415 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c8963f42-d594-4be6-a4da-930a8bf3d4cc tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:54.770024 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=88,nova_cell1:UPDATE=7,nova_cell1:INSERT=5 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:07:54.963472 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-c8963f42-d594-4be6-a4da-930a8bf3d4cc tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:54.969821 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-c8963f42-d594-4be6-a4da-930a8bf3d4cc tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1723 microversion: 2.1 time: 0.258127 Jul 19 11:07:54.970415 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 289/555] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:54 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1723 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:55.079059 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-3bd5c684-0cdc-42b8-a52e-2dcc07f28e2f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:55.079866 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-3bd5c684-0cdc-42b8-a52e-2dcc07f28e2f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:55.087109 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3bd5c684-0cdc-42b8-a52e-2dcc07f28e2f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:55.087830 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3bd5c684-0cdc-42b8-a52e-2dcc07f28e2f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:55.088605 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3bd5c684-0cdc-42b8-a52e-2dcc07f28e2f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:55.089078 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3bd5c684-0cdc-42b8-a52e-2dcc07f28e2f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:55.265532 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-3bd5c684-0cdc-42b8-a52e-2dcc07f28e2f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:55.289365 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-3bd5c684-0cdc-42b8-a52e-2dcc07f28e2f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:55.290886 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-3bd5c684-0cdc-42b8-a52e-2dcc07f28e2f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:55.292069 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-3bd5c684-0cdc-42b8-a52e-2dcc07f28e2f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1751 microversion: 2.20 time: 0.217068 Jul 19 11:07:55.292678 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 267/556] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:55 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1751 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:55.693441 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-cd4d0f3d-b0a4-4b70-bff1-23520ef9ef41 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:55.695012 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-cd4d0f3d-b0a4-4b70-bff1-23520ef9ef41 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:55.711507 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cd4d0f3d-b0a4-4b70-bff1-23520ef9ef41 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:55.711971 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cd4d0f3d-b0a4-4b70-bff1-23520ef9ef41 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:55.718203 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cd4d0f3d-b0a4-4b70-bff1-23520ef9ef41 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:55.719279 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cd4d0f3d-b0a4-4b70-bff1-23520ef9ef41 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:55.986002 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-cd4d0f3d-b0a4-4b70-bff1-23520ef9ef41 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:55.989259 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-9796182e-4a70-415c-b198-6b9b5bf34c89 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:55.990622 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-9796182e-4a70-415c-b198-6b9b5bf34c89 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:55.998679 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-cd4d0f3d-b0a4-4b70-bff1-23520ef9ef41 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1358 microversion: 2.1 time: 0.309026 Jul 19 11:07:56.003089 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 290/557] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:55 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1358 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:56.004098 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9796182e-4a70-415c-b198-6b9b5bf34c89 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:56.004562 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9796182e-4a70-415c-b198-6b9b5bf34c89 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:56.011839 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9796182e-4a70-415c-b198-6b9b5bf34c89 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:56.012558 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9796182e-4a70-415c-b198-6b9b5bf34c89 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:56.276967 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-9796182e-4a70-415c-b198-6b9b5bf34c89 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:56.287504 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-9796182e-4a70-415c-b198-6b9b5bf34c89 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1723 microversion: 2.1 time: 0.303592 Jul 19 11:07:56.288567 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 268/558] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:55 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1723 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:56.310090 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-d19b1e38-98a4-424b-b9bb-b402904377f6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:56.311069 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-d19b1e38-98a4-424b-b9bb-b402904377f6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:56.322791 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d19b1e38-98a4-424b-b9bb-b402904377f6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:56.323610 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d19b1e38-98a4-424b-b9bb-b402904377f6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:56.326813 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d19b1e38-98a4-424b-b9bb-b402904377f6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:56.327635 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d19b1e38-98a4-424b-b9bb-b402904377f6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:56.498813 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-d19b1e38-98a4-424b-b9bb-b402904377f6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:56.511470 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-d19b1e38-98a4-424b-b9bb-b402904377f6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:56.514030 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-d19b1e38-98a4-424b-b9bb-b402904377f6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:56.515858 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-d19b1e38-98a4-424b-b9bb-b402904377f6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1751 microversion: 2.20 time: 0.209770 Jul 19 11:07:56.516795 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 291/559] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:56 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1751 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:07:57.018022 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-dd9985fb-c3e1-45b7-99a1-801789cd7a36 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:57.018752 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-dd9985fb-c3e1-45b7-99a1-801789cd7a36 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:57.030180 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dd9985fb-c3e1-45b7-99a1-801789cd7a36 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:57.030968 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dd9985fb-c3e1-45b7-99a1-801789cd7a36 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:57.031941 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dd9985fb-c3e1-45b7-99a1-801789cd7a36 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:57.032557 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dd9985fb-c3e1-45b7-99a1-801789cd7a36 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:57.293947 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-dd9985fb-c3e1-45b7-99a1-801789cd7a36 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:57.312924 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-2d82f512-0984-496d-9e5a-9559b5a49bb4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:57.314386 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-dd9985fb-c3e1-45b7-99a1-801789cd7a36 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1358 microversion: 2.1 time: 0.295941 Jul 19 11:07:57.316438 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-2d82f512-0984-496d-9e5a-9559b5a49bb4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:57.328770 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 269/560] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:57 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1358 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:57.358453 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2d82f512-0984-496d-9e5a-9559b5a49bb4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:57.359315 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2d82f512-0984-496d-9e5a-9559b5a49bb4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:57.361216 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2d82f512-0984-496d-9e5a-9559b5a49bb4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:57.363051 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2d82f512-0984-496d-9e5a-9559b5a49bb4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:57.537416 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-61b9a88b-244a-4c08-8a5b-4b92eb85e312 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:57.538066 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-61b9a88b-244a-4c08-8a5b-4b92eb85e312 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:57.557276 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-61b9a88b-244a-4c08-8a5b-4b92eb85e312 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:57.557735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-61b9a88b-244a-4c08-8a5b-4b92eb85e312 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:57.558521 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-61b9a88b-244a-4c08-8a5b-4b92eb85e312 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:57.559067 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-61b9a88b-244a-4c08-8a5b-4b92eb85e312 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:57.919603 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-61b9a88b-244a-4c08-8a5b-4b92eb85e312 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:57.949867 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-61b9a88b-244a-4c08-8a5b-4b92eb85e312 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:57.951704 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-61b9a88b-244a-4c08-8a5b-4b92eb85e312 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:57.952944 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-61b9a88b-244a-4c08-8a5b-4b92eb85e312 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1751 microversion: 2.20 time: 0.421362 Jul 19 11:07:57.953849 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 270/561] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:57 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1751 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:07:58.066952 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-2d82f512-0984-496d-9e5a-9559b5a49bb4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:58.086750 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-2d82f512-0984-496d-9e5a-9559b5a49bb4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1723 microversion: 2.1 time: 0.785521 Jul 19 11:07:58.094761 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 292/562] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:57 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1723 bytes in 793 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:58.353240 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-922204b9-e1f2-41af-ad17-feeb44cb2c55 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:58.354959 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-922204b9-e1f2-41af-ad17-feeb44cb2c55 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:58.365740 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell0:SELECT=2 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:07:58.369116 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell0:SELECT=10 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:07:58.374235 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-922204b9-e1f2-41af-ad17-feeb44cb2c55 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:58.375900 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-922204b9-e1f2-41af-ad17-feeb44cb2c55 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:58.380289 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-922204b9-e1f2-41af-ad17-feeb44cb2c55 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:58.380998 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-922204b9-e1f2-41af-ad17-feeb44cb2c55 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:58.428383 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:07:58.704302 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-922204b9-e1f2-41af-ad17-feeb44cb2c55 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:58.738660 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-922204b9-e1f2-41af-ad17-feeb44cb2c55 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1358 microversion: 2.1 time: 0.405386 Jul 19 11:07:58.739639 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 271/563] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:58 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1358 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:58.977090 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-c67607ca-d93a-46a8-af1d-8b4edc503122 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:58.978465 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-c67607ca-d93a-46a8-af1d-8b4edc503122 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:58.988741 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell0:SELECT=2 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:07:58.990681 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c67607ca-d93a-46a8-af1d-8b4edc503122 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:58.991832 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c67607ca-d93a-46a8-af1d-8b4edc503122 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:58.997992 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c67607ca-d93a-46a8-af1d-8b4edc503122 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:58.999275 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c67607ca-d93a-46a8-af1d-8b4edc503122 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:59.105121 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-99b2022b-b651-483f-9fb8-09dae6ef9aa4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:59.105732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-99b2022b-b651-483f-9fb8-09dae6ef9aa4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:59.111948 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-99b2022b-b651-483f-9fb8-09dae6ef9aa4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:59.112402 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-99b2022b-b651-483f-9fb8-09dae6ef9aa4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:59.117502 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-99b2022b-b651-483f-9fb8-09dae6ef9aa4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:59.117960 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-99b2022b-b651-483f-9fb8-09dae6ef9aa4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:59.439629 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-99b2022b-b651-483f-9fb8-09dae6ef9aa4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:59.454620 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-c67607ca-d93a-46a8-af1d-8b4edc503122 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:59.455488 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-99b2022b-b651-483f-9fb8-09dae6ef9aa4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1723 microversion: 2.1 time: 0.349817 Jul 19 11:07:59.470049 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-c67607ca-d93a-46a8-af1d-8b4edc503122 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:59.471827 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-c67607ca-d93a-46a8-af1d-8b4edc503122 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:07:59.473433 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 272/564] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:59 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1723 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:07:59.474202 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-c67607ca-d93a-46a8-af1d-8b4edc503122 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1751 microversion: 2.20 time: 0.505281 Jul 19 11:07:59.475230 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 293/565] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:07:58 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1751 bytes in 508 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:07:59.755406 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-aac7e303-34a1-4dfd-acdc-f1a23d57b0f5 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:07:59.755941 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-aac7e303-34a1-4dfd-acdc-f1a23d57b0f5 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:07:59.767752 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-aac7e303-34a1-4dfd-acdc-f1a23d57b0f5 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:59.768220 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-aac7e303-34a1-4dfd-acdc-f1a23d57b0f5 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:07:59.774699 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-aac7e303-34a1-4dfd-acdc-f1a23d57b0f5 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:07:59.775208 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-aac7e303-34a1-4dfd-acdc-f1a23d57b0f5 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:00.222827 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-aac7e303-34a1-4dfd-acdc-f1a23d57b0f5 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:00.230888 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-aac7e303-34a1-4dfd-acdc-f1a23d57b0f5 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1358 microversion: 2.1 time: 0.479474 Jul 19 11:08:00.231479 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 273/566] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:07:59 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1358 bytes in 481 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:00.472728 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-e4a217c7-7ec5-4d0c-a22e-dcedb8117bb4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:00.474181 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-e4a217c7-7ec5-4d0c-a22e-dcedb8117bb4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:00.487362 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e4a217c7-7ec5-4d0c-a22e-dcedb8117bb4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:00.492939 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-4a769854-7fa5-47ae-8e8d-9afcc13dc542 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:00.494167 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-4a769854-7fa5-47ae-8e8d-9afcc13dc542 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:00.498002 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e4a217c7-7ec5-4d0c-a22e-dcedb8117bb4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:00.499105 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e4a217c7-7ec5-4d0c-a22e-dcedb8117bb4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:00.502457 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e4a217c7-7ec5-4d0c-a22e-dcedb8117bb4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:00.510592 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4a769854-7fa5-47ae-8e8d-9afcc13dc542 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:00.511369 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4a769854-7fa5-47ae-8e8d-9afcc13dc542 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:00.512391 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4a769854-7fa5-47ae-8e8d-9afcc13dc542 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:00.517978 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4a769854-7fa5-47ae-8e8d-9afcc13dc542 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:00.888475 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-e4a217c7-7ec5-4d0c-a22e-dcedb8117bb4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:00.923274 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-e4a217c7-7ec5-4d0c-a22e-dcedb8117bb4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1723 microversion: 2.1 time: 0.454332 Jul 19 11:08:00.935973 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 294/567] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:00 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1723 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:01.052328 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-4a769854-7fa5-47ae-8e8d-9afcc13dc542 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:01.061681 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-4a769854-7fa5-47ae-8e8d-9afcc13dc542 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:01.067014 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-4a769854-7fa5-47ae-8e8d-9afcc13dc542 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:01.070636 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-4a769854-7fa5-47ae-8e8d-9afcc13dc542 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1751 microversion: 2.20 time: 0.581796 Jul 19 11:08:01.075710 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 274/568] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:08:00 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1751 bytes in 587 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:08:01.250708 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-42b50fe0-cf35-4279-9e90-ae6408e0fd81 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:01.254065 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-42b50fe0-cf35-4279-9e90-ae6408e0fd81 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:01.263043 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-42b50fe0-cf35-4279-9e90-ae6408e0fd81 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:01.263620 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-42b50fe0-cf35-4279-9e90-ae6408e0fd81 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:01.264599 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-42b50fe0-cf35-4279-9e90-ae6408e0fd81 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:01.265196 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-42b50fe0-cf35-4279-9e90-ae6408e0fd81 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:01.418673 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-42b50fe0-cf35-4279-9e90-ae6408e0fd81 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:01.426468 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-42b50fe0-cf35-4279-9e90-ae6408e0fd81 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1358 microversion: 2.1 time: 0.182928 Jul 19 11:08:01.427133 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 295/569] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:01 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1358 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:01.957008 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-739bb590-17f4-4d25-8ebd-0d4122048316 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:01.959579 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-739bb590-17f4-4d25-8ebd-0d4122048316 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:01.967291 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-739bb590-17f4-4d25-8ebd-0d4122048316 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:01.968162 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-739bb590-17f4-4d25-8ebd-0d4122048316 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:01.968933 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-739bb590-17f4-4d25-8ebd-0d4122048316 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:01.969359 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-739bb590-17f4-4d25-8ebd-0d4122048316 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:02.095550 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-08219b33-92b1-4532-b535-2120200bb4b2 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:02.096169 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-08219b33-92b1-4532-b535-2120200bb4b2 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:02.115317 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-08219b33-92b1-4532-b535-2120200bb4b2 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:02.115780 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-08219b33-92b1-4532-b535-2120200bb4b2 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:02.117146 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-08219b33-92b1-4532-b535-2120200bb4b2 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:02.117576 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-08219b33-92b1-4532-b535-2120200bb4b2 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:02.207228 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-739bb590-17f4-4d25-8ebd-0d4122048316 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:02.219457 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-739bb590-17f4-4d25-8ebd-0d4122048316 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1723 microversion: 2.1 time: 0.269852 Jul 19 11:08:02.221144 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 275/570] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:01 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1723 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:02.416008 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-08219b33-92b1-4532-b535-2120200bb4b2 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:02.428286 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-08219b33-92b1-4532-b535-2120200bb4b2 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:02.433012 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-08219b33-92b1-4532-b535-2120200bb4b2 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:02.437668 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-08219b33-92b1-4532-b535-2120200bb4b2 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1751 microversion: 2.20 time: 0.346962 Jul 19 11:08:02.443391 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-782994e7-77f9-4b4f-92dd-812469e2443f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:02.444344 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-782994e7-77f9-4b4f-92dd-812469e2443f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:02.452616 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 296/571] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:08:02 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1751 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:08:02.463873 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-782994e7-77f9-4b4f-92dd-812469e2443f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:02.463873 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-782994e7-77f9-4b4f-92dd-812469e2443f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:02.463873 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-782994e7-77f9-4b4f-92dd-812469e2443f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:02.463873 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-782994e7-77f9-4b4f-92dd-812469e2443f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:02.852732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-782994e7-77f9-4b4f-92dd-812469e2443f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:02.879078 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-782994e7-77f9-4b4f-92dd-812469e2443f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1358 microversion: 2.1 time: 0.439371 Jul 19 11:08:02.892729 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 276/572] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:02 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1358 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:08:03.100876 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-e7ca5e3e-7d72-4122-ad43-21c0b090d90b req-e6529734-4203-4230-ae63-e7bc522915d3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "03624642-ce02-450f-821d-1c135e5d7a4a", "tag": "6b6d7587-f2e5-4c60-8cf8-d1103dc5c0b6"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:08:03.129800 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-e7ca5e3e-7d72-4122-ad43-21c0b090d90b req-e6529734-4203-4230-ae63-e7bc522915d3 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:03.131516 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-e7ca5e3e-7d72-4122-ad43-21c0b090d90b req-e6529734-4203-4230-ae63-e7bc522915d3 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:03.140763 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-29432aeb-6181-4a7a-ae1c-aafb9733fa42 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Action: 'create', calling method: .version_select at 0x7f04aa3ad9d0>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-1205270003", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDFLPzSr1l+E4RaFQcHNmGb9KyYXNb7a5BV7zWav2uoUJRGZx8MWHfu4Jp7klESFEyvuuWxhV0kUjnw4pYLcoWvAlO9kgV/5xSHw4c96cI/rC9KZrcPUyJOFNJhYF4E0dg=="}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:08:03.159257 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.quota [None req-29432aeb-6181-4a7a-ae1c-aafb9733fa42 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Getting quotas for project a7109adf88ac4391aee35c150e96e0b1. Resources: {'key_pairs'} {{(pid=117848) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:08:03.167214 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.quota [None req-29432aeb-6181-4a7a-ae1c-aafb9733fa42 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Getting quotas for user a217e983effa43f3a4f84221984189c5 and project a7109adf88ac4391aee35c150e96e0b1. Resources: {'key_pairs'} {{(pid=117848) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:08:03.169568 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-e7ca5e3e-7d72-4122-ad43-21c0b090d90b req-e6529734-4203-4230-ae63-e7bc522915d3 service nova] Creating event network-changed:6b6d7587-f2e5-4c60-8cf8-d1103dc5c0b6 for instance 03624642-ce02-450f-821d-1c135e5d7a4a on hv-cinder-maste Jul 19 11:08:03.175577 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-e7ca5e3e-7d72-4122-ad43-21c0b090d90b req-e6529734-4203-4230-ae63-e7bc522915d3 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.080364 Jul 19 11:08:03.176480 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 277/573] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:08:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:08:03.205880 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-29432aeb-6181-4a7a-ae1c-aafb9733fa42 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "POST /compute/v2.1/os-keypairs" status: 200 len: 400 microversion: 2.1 time: 0.315764 Jul 19 11:08:03.207777 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 297/574] 10.222.0.33 () {60 vars in 1158 bytes} [Tue Jul 19 11:08:02 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:08:03.242231 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-468c1782-8066-4cb3-bb93-992cf027e25e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:03.243660 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-468c1782-8066-4cb3-bb93-992cf027e25e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:03.252227 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-468c1782-8066-4cb3-bb93-992cf027e25e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:03.253051 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-468c1782-8066-4cb3-bb93-992cf027e25e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:03.254128 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-468c1782-8066-4cb3-bb93-992cf027e25e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:03.254831 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-468c1782-8066-4cb3-bb93-992cf027e25e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:03.477404 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-a2903ab9-c97c-4d64-a6ce-18206eabe54c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:03.480192 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-a2903ab9-c97c-4d64-a6ce-18206eabe54c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:03.488392 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a2903ab9-c97c-4d64-a6ce-18206eabe54c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:03.492579 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a2903ab9-c97c-4d64-a6ce-18206eabe54c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:03.493701 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a2903ab9-c97c-4d64-a6ce-18206eabe54c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:03.494536 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a2903ab9-c97c-4d64-a6ce-18206eabe54c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:03.526594 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-468c1782-8066-4cb3-bb93-992cf027e25e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:03.541312 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-468c1782-8066-4cb3-bb93-992cf027e25e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1723 microversion: 2.1 time: 0.307147 Jul 19 11:08:03.544073 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 278/575] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:03 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1723 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:03.763271 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-a2903ab9-c97c-4d64-a6ce-18206eabe54c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:03.771958 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-a2903ab9-c97c-4d64-a6ce-18206eabe54c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:03.774440 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-a2903ab9-c97c-4d64-a6ce-18206eabe54c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:03.775842 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-a2903ab9-c97c-4d64-a6ce-18206eabe54c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1807 microversion: 2.20 time: 0.309337 Jul 19 11:08:03.776620 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 298/576] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:08:03 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1807 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:08:03.898407 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-3e6f278e-0878-498c-b668-a6ebc1fda663 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:03.899359 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-3e6f278e-0878-498c-b668-a6ebc1fda663 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:03.909821 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3e6f278e-0878-498c-b668-a6ebc1fda663 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:03.910929 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3e6f278e-0878-498c-b668-a6ebc1fda663 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:03.911732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3e6f278e-0878-498c-b668-a6ebc1fda663 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:03.912634 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3e6f278e-0878-498c-b668-a6ebc1fda663 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:04.117684 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-3e6f278e-0878-498c-b668-a6ebc1fda663 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:04.127594 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-3e6f278e-0878-498c-b668-a6ebc1fda663 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1358 microversion: 2.1 time: 0.233972 Jul 19 11:08:04.128462 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 279/577] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:03 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1358 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:04.563259 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-628e32df-b722-477b-b4c2-ce4ca491e803 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:04.564112 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-628e32df-b722-477b-b4c2-ce4ca491e803 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:04.572226 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-628e32df-b722-477b-b4c2-ce4ca491e803 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:04.572807 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-628e32df-b722-477b-b4c2-ce4ca491e803 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:04.573696 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-628e32df-b722-477b-b4c2-ce4ca491e803 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:04.574364 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-628e32df-b722-477b-b4c2-ce4ca491e803 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:04.721699 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-628e32df-b722-477b-b4c2-ce4ca491e803 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:04.730288 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-628e32df-b722-477b-b4c2-ce4ca491e803 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1723 microversion: 2.1 time: 0.172813 Jul 19 11:08:04.730850 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 299/578] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:04 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1723 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:04.794388 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-3efb6082-d058-416e-96d9-622c07f38eed tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:04.795022 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-3efb6082-d058-416e-96d9-622c07f38eed tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:04.801133 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3efb6082-d058-416e-96d9-622c07f38eed tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:04.801587 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3efb6082-d058-416e-96d9-622c07f38eed tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:04.802350 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3efb6082-d058-416e-96d9-622c07f38eed tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:04.802785 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3efb6082-d058-416e-96d9-622c07f38eed tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:04.942336 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-3efb6082-d058-416e-96d9-622c07f38eed tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:04.951185 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-3efb6082-d058-416e-96d9-622c07f38eed tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:04.952805 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-3efb6082-d058-416e-96d9-622c07f38eed tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:04.954032 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-3efb6082-d058-416e-96d9-622c07f38eed tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1807 microversion: 2.20 time: 0.163484 Jul 19 11:08:04.954681 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 280/579] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:08:04 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1807 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:08:05.145248 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-691008b6-4d98-4f0d-85e6-7e089f2b553b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:05.145848 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-691008b6-4d98-4f0d-85e6-7e089f2b553b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:05.152329 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-691008b6-4d98-4f0d-85e6-7e089f2b553b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:05.152812 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-691008b6-4d98-4f0d-85e6-7e089f2b553b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:05.153562 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-691008b6-4d98-4f0d-85e6-7e089f2b553b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:05.153997 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-691008b6-4d98-4f0d-85e6-7e089f2b553b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:05.318890 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-691008b6-4d98-4f0d-85e6-7e089f2b553b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:05.327502 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-691008b6-4d98-4f0d-85e6-7e089f2b553b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1358 microversion: 2.1 time: 0.186016 Jul 19 11:08:05.328079 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 300/580] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:05 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1358 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:05.750118 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-71d75d9f-b9f8-43c4-86dd-729a5a343143 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:05.751284 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-71d75d9f-b9f8-43c4-86dd-729a5a343143 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:05.762977 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-71d75d9f-b9f8-43c4-86dd-729a5a343143 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:05.763649 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-71d75d9f-b9f8-43c4-86dd-729a5a343143 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:05.764595 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-71d75d9f-b9f8-43c4-86dd-729a5a343143 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:05.765420 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-71d75d9f-b9f8-43c4-86dd-729a5a343143 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:05.974877 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-148fc967-08ee-44f1-ace1-42c59084c2f7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:05.977231 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-148fc967-08ee-44f1-ace1-42c59084c2f7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:05.993192 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-148fc967-08ee-44f1-ace1-42c59084c2f7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:05.993749 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-148fc967-08ee-44f1-ace1-42c59084c2f7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:05.997564 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-148fc967-08ee-44f1-ace1-42c59084c2f7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:05.998089 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-148fc967-08ee-44f1-ace1-42c59084c2f7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:06.134369 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-71d75d9f-b9f8-43c4-86dd-729a5a343143 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:06.161778 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-71d75d9f-b9f8-43c4-86dd-729a5a343143 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1723 microversion: 2.1 time: 0.415781 Jul 19 11:08:06.172227 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 281/581] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:05 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1723 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:06.298038 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-148fc967-08ee-44f1-ace1-42c59084c2f7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:06.309878 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-148fc967-08ee-44f1-ace1-42c59084c2f7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:06.311559 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-148fc967-08ee-44f1-ace1-42c59084c2f7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:06.312850 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-148fc967-08ee-44f1-ace1-42c59084c2f7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1807 microversion: 2.20 time: 0.344977 Jul 19 11:08:06.313654 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 301/582] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:08:05 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1807 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:08:06.345117 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-82506732-3dfd-4d4c-8cd8-c7714378cff8 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:06.345743 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-82506732-3dfd-4d4c-8cd8-c7714378cff8 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:06.361741 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-82506732-3dfd-4d4c-8cd8-c7714378cff8 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:06.362196 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-82506732-3dfd-4d4c-8cd8-c7714378cff8 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:06.362989 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-82506732-3dfd-4d4c-8cd8-c7714378cff8 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:06.363429 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-82506732-3dfd-4d4c-8cd8-c7714378cff8 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:06.667051 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-82506732-3dfd-4d4c-8cd8-c7714378cff8 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:06.691412 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-82506732-3dfd-4d4c-8cd8-c7714378cff8 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1358 microversion: 2.1 time: 0.349853 Jul 19 11:08:06.692623 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 282/583] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:06 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1358 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:07.179388 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-3ed7f4c7-90cf-4114-8bd1-a4c3c3cf86ae tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:07.180003 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-3ed7f4c7-90cf-4114-8bd1-a4c3c3cf86ae tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:07.186589 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3ed7f4c7-90cf-4114-8bd1-a4c3c3cf86ae tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:07.187020 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3ed7f4c7-90cf-4114-8bd1-a4c3c3cf86ae tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:07.187781 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3ed7f4c7-90cf-4114-8bd1-a4c3c3cf86ae tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:07.188207 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3ed7f4c7-90cf-4114-8bd1-a4c3c3cf86ae tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:07.341293 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-b2dd2ebd-5cb0-45a2-8c6d-dcecc263b8df tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:07.343052 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-b2dd2ebd-5cb0-45a2-8c6d-dcecc263b8df tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:07.361956 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b2dd2ebd-5cb0-45a2-8c6d-dcecc263b8df tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:07.362795 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b2dd2ebd-5cb0-45a2-8c6d-dcecc263b8df tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:07.367877 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b2dd2ebd-5cb0-45a2-8c6d-dcecc263b8df tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:07.368546 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b2dd2ebd-5cb0-45a2-8c6d-dcecc263b8df tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:07.395784 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-3ed7f4c7-90cf-4114-8bd1-a4c3c3cf86ae tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:07.416132 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-3ed7f4c7-90cf-4114-8bd1-a4c3c3cf86ae tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1723 microversion: 2.1 time: 0.240521 Jul 19 11:08:07.421379 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 302/584] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:07 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1723 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:07.565951 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b2dd2ebd-5cb0-45a2-8c6d-dcecc263b8df tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:07.592389 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b2dd2ebd-5cb0-45a2-8c6d-dcecc263b8df tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:07.594011 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b2dd2ebd-5cb0-45a2-8c6d-dcecc263b8df tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:07.595378 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-b2dd2ebd-5cb0-45a2-8c6d-dcecc263b8df tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1807 microversion: 2.20 time: 0.264122 Jul 19 11:08:07.596438 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 283/585] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:08:07 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1807 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:08:07.708725 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-dc084826-eaaa-4254-b033-8ac865166d38 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:07.710315 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-dc084826-eaaa-4254-b033-8ac865166d38 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:07.712915 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_api:SELECT=73,nova_api:INSERT=1 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:08:07.729311 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-2ec0c524-dacd-4997-a0b6-c4671f763f86 req-4d37bfdd-484e-43e4-8ebc-f34432f569ae service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "03624642-ce02-450f-821d-1c135e5d7a4a", "name": "network-vif-plugged", "status": "completed", "tag": "6b6d7587-f2e5-4c60-8cf8-d1103dc5c0b6"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:08:07.731101 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dc084826-eaaa-4254-b033-8ac865166d38 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:07.731778 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dc084826-eaaa-4254-b033-8ac865166d38 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:07.732819 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dc084826-eaaa-4254-b033-8ac865166d38 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:07.733462 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dc084826-eaaa-4254-b033-8ac865166d38 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:07.744055 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-2ec0c524-dacd-4997-a0b6-c4671f763f86 req-4d37bfdd-484e-43e4-8ebc-f34432f569ae service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:07.746188 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-2ec0c524-dacd-4997-a0b6-c4671f763f86 req-4d37bfdd-484e-43e4-8ebc-f34432f569ae service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:07.771182 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-2ec0c524-dacd-4997-a0b6-c4671f763f86 req-4d37bfdd-484e-43e4-8ebc-f34432f569ae service nova] Creating event network-vif-plugged:6b6d7587-f2e5-4c60-8cf8-d1103dc5c0b6 for instance 03624642-ce02-450f-821d-1c135e5d7a4a on hv-cinder-maste Jul 19 11:08:07.790782 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-2ec0c524-dacd-4997-a0b6-c4671f763f86 req-4d37bfdd-484e-43e4-8ebc-f34432f569ae service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.066770 Jul 19 11:08:07.794970 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 284/586] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:08:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:08:08.044332 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-dc084826-eaaa-4254-b033-8ac865166d38 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:08.083384 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-dc084826-eaaa-4254-b033-8ac865166d38 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1358 microversion: 2.1 time: 0.371213 Jul 19 11:08:08.084135 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 303/587] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:07 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1358 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:08.439611 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-0a810d91-43f3-4fa0-a0cd-de38a29331bf tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:08.441338 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-0a810d91-43f3-4fa0-a0cd-de38a29331bf tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:08.452793 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0a810d91-43f3-4fa0-a0cd-de38a29331bf tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:08.457510 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0a810d91-43f3-4fa0-a0cd-de38a29331bf tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:08.458827 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0a810d91-43f3-4fa0-a0cd-de38a29331bf tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:08.459506 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0a810d91-43f3-4fa0-a0cd-de38a29331bf tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:08.615095 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-c9b6b16a-8479-4660-b460-8329700d10de tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:08.616684 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-c9b6b16a-8479-4660-b460-8329700d10de tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:08.637456 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c9b6b16a-8479-4660-b460-8329700d10de tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:08.640782 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c9b6b16a-8479-4660-b460-8329700d10de tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:08.642219 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c9b6b16a-8479-4660-b460-8329700d10de tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:08.644120 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c9b6b16a-8479-4660-b460-8329700d10de tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:08.786468 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-0a810d91-43f3-4fa0-a0cd-de38a29331bf tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:08.800175 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-0a810d91-43f3-4fa0-a0cd-de38a29331bf tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1723 microversion: 2.1 time: 0.364888 Jul 19 11:08:08.801462 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 285/588] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:08 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1723 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:08.888913 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-c9b6b16a-8479-4660-b460-8329700d10de tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:08.907704 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-c9b6b16a-8479-4660-b460-8329700d10de tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:08.910025 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-c9b6b16a-8479-4660-b460-8329700d10de tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:08.911464 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-c9b6b16a-8479-4660-b460-8329700d10de tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1807 microversion: 2.20 time: 0.301206 Jul 19 11:08:08.912665 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 304/589] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:08:08 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1807 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:08:09.099840 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-4af59946-0c6e-4edc-a834-803a925c746a tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:09.100484 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-4af59946-0c6e-4edc-a834-803a925c746a tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:09.106790 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4af59946-0c6e-4edc-a834-803a925c746a tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:09.107237 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4af59946-0c6e-4edc-a834-803a925c746a tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:09.108255 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4af59946-0c6e-4edc-a834-803a925c746a tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:09.108715 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4af59946-0c6e-4edc-a834-803a925c746a tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:09.276077 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-4af59946-0c6e-4edc-a834-803a925c746a tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:09.283491 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-4af59946-0c6e-4edc-a834-803a925c746a tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1358 microversion: 2.1 time: 0.187570 Jul 19 11:08:09.284039 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 286/590] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:09 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1358 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:09.825092 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-ad941e3b-a960-4304-8504-22816f54897c tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:09.829256 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-ad941e3b-a960-4304-8504-22816f54897c tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:09.847512 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ad941e3b-a960-4304-8504-22816f54897c tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:09.847991 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ad941e3b-a960-4304-8504-22816f54897c tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:09.849210 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ad941e3b-a960-4304-8504-22816f54897c tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:09.849685 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ad941e3b-a960-4304-8504-22816f54897c tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:09.930478 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-108b10ee-30d4-45d1-9993-732e4af48b64 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:09.931703 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-108b10ee-30d4-45d1-9993-732e4af48b64 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:09.945922 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-108b10ee-30d4-45d1-9993-732e4af48b64 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:09.946836 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-108b10ee-30d4-45d1-9993-732e4af48b64 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:09.948768 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-108b10ee-30d4-45d1-9993-732e4af48b64 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:09.949387 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-108b10ee-30d4-45d1-9993-732e4af48b64 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:10.124750 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-ad941e3b-a960-4304-8504-22816f54897c tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:10.133775 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-ad941e3b-a960-4304-8504-22816f54897c tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1723 microversion: 2.1 time: 0.319317 Jul 19 11:08:10.135795 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 305/591] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:09 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1723 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:10.203855 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-108b10ee-30d4-45d1-9993-732e4af48b64 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:10.213347 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-108b10ee-30d4-45d1-9993-732e4af48b64 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:10.214939 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-108b10ee-30d4-45d1-9993-732e4af48b64 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:10.216384 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-108b10ee-30d4-45d1-9993-732e4af48b64 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1807 microversion: 2.20 time: 0.289930 Jul 19 11:08:10.217399 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 287/592] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:08:09 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1807 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:08:10.301236 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-79d8508c-86ef-4291-a623-1698fc201c3f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:10.301994 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-79d8508c-86ef-4291-a623-1698fc201c3f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:10.308952 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-79d8508c-86ef-4291-a623-1698fc201c3f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:10.309404 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-79d8508c-86ef-4291-a623-1698fc201c3f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:10.310184 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-79d8508c-86ef-4291-a623-1698fc201c3f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:10.310623 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-79d8508c-86ef-4291-a623-1698fc201c3f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:10.462696 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-79d8508c-86ef-4291-a623-1698fc201c3f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:10.470605 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-79d8508c-86ef-4291-a623-1698fc201c3f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1358 microversion: 2.1 time: 0.173688 Jul 19 11:08:10.481425 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 306/593] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:10 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1358 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:11.152522 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-d9e9de4a-18b3-4b8a-87dc-3d4a067c7f20 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:11.153667 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-d9e9de4a-18b3-4b8a-87dc-3d4a067c7f20 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:11.159923 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d9e9de4a-18b3-4b8a-87dc-3d4a067c7f20 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:11.160384 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d9e9de4a-18b3-4b8a-87dc-3d4a067c7f20 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:11.161309 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d9e9de4a-18b3-4b8a-87dc-3d4a067c7f20 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:11.161714 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d9e9de4a-18b3-4b8a-87dc-3d4a067c7f20 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:11.235446 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-d78e58a0-e301-4209-8bae-372ad5768535 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:11.239655 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-d78e58a0-e301-4209-8bae-372ad5768535 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:11.251908 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d78e58a0-e301-4209-8bae-372ad5768535 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:11.252353 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d78e58a0-e301-4209-8bae-372ad5768535 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:11.253178 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d78e58a0-e301-4209-8bae-372ad5768535 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:11.253610 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d78e58a0-e301-4209-8bae-372ad5768535 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:11.270593 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=100 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:08:11.315560 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-d9e9de4a-18b3-4b8a-87dc-3d4a067c7f20 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:11.332844 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-d9e9de4a-18b3-4b8a-87dc-3d4a067c7f20 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1723 microversion: 2.1 time: 0.180383 Jul 19 11:08:11.337178 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 288/594] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:11 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1723 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:11.439375 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-d78e58a0-e301-4209-8bae-372ad5768535 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:11.448396 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-d78e58a0-e301-4209-8bae-372ad5768535 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:11.449929 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-d78e58a0-e301-4209-8bae-372ad5768535 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:11.451096 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-d78e58a0-e301-4209-8bae-372ad5768535 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1807 microversion: 2.20 time: 0.219634 Jul 19 11:08:11.451654 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 307/595] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:08:11 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1807 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:08:11.488525 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-7c5cc022-d363-448e-8386-c0b9db1d015b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:11.489108 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-7c5cc022-d363-448e-8386-c0b9db1d015b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:11.503554 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7c5cc022-d363-448e-8386-c0b9db1d015b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:11.503554 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7c5cc022-d363-448e-8386-c0b9db1d015b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:11.503554 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7c5cc022-d363-448e-8386-c0b9db1d015b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:11.503554 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7c5cc022-d363-448e-8386-c0b9db1d015b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:11.533408 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=100 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:08:11.659694 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-7c5cc022-d363-448e-8386-c0b9db1d015b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:11.667868 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-7c5cc022-d363-448e-8386-c0b9db1d015b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1358 microversion: 2.1 time: 0.183162 Jul 19 11:08:11.668450 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 289/596] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:11 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1358 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:12.350034 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-56bd1ce8-20f1-45a7-981e-5fd5ec8f4be4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:12.351469 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-56bd1ce8-20f1-45a7-981e-5fd5ec8f4be4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:12.358947 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-56bd1ce8-20f1-45a7-981e-5fd5ec8f4be4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:12.359493 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-56bd1ce8-20f1-45a7-981e-5fd5ec8f4be4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:12.360362 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-56bd1ce8-20f1-45a7-981e-5fd5ec8f4be4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:12.360903 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-56bd1ce8-20f1-45a7-981e-5fd5ec8f4be4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:12.470508 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-254bd44c-6ee2-47a6-95ae-a273b2972741 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:12.471209 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-254bd44c-6ee2-47a6-95ae-a273b2972741 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:12.478431 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-254bd44c-6ee2-47a6-95ae-a273b2972741 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:12.478908 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-254bd44c-6ee2-47a6-95ae-a273b2972741 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:12.479846 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-254bd44c-6ee2-47a6-95ae-a273b2972741 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:12.480285 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-254bd44c-6ee2-47a6-95ae-a273b2972741 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:12.514240 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-56bd1ce8-20f1-45a7-981e-5fd5ec8f4be4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:12.525018 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-56bd1ce8-20f1-45a7-981e-5fd5ec8f4be4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1723 microversion: 2.1 time: 0.179001 Jul 19 11:08:12.535795 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 308/597] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:12 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1723 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:12.643957 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-254bd44c-6ee2-47a6-95ae-a273b2972741 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:12.657457 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-254bd44c-6ee2-47a6-95ae-a273b2972741 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:12.658982 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-254bd44c-6ee2-47a6-95ae-a273b2972741 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:12.660179 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-254bd44c-6ee2-47a6-95ae-a273b2972741 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1807 microversion: 2.20 time: 0.194579 Jul 19 11:08:12.660754 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 290/598] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:08:12 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1807 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:08:12.685298 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-6f8b9f2d-2c97-4147-b222-568b425e7397 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:12.685901 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-6f8b9f2d-2c97-4147-b222-568b425e7397 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:12.691889 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6f8b9f2d-2c97-4147-b222-568b425e7397 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:12.692329 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6f8b9f2d-2c97-4147-b222-568b425e7397 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:12.693104 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6f8b9f2d-2c97-4147-b222-568b425e7397 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:12.693536 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6f8b9f2d-2c97-4147-b222-568b425e7397 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:12.835445 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-6f8b9f2d-2c97-4147-b222-568b425e7397 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:12.843070 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-6f8b9f2d-2c97-4147-b222-568b425e7397 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1358 microversion: 2.1 time: 0.161596 Jul 19 11:08:12.843635 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 309/599] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:12 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1358 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:13.544119 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-f4dd7e87-bbf6-454b-b8e3-005051d8d426 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:13.544936 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-f4dd7e87-bbf6-454b-b8e3-005051d8d426 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:13.556545 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f4dd7e87-bbf6-454b-b8e3-005051d8d426 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:13.557045 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f4dd7e87-bbf6-454b-b8e3-005051d8d426 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:13.557841 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f4dd7e87-bbf6-454b-b8e3-005051d8d426 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:13.558307 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f4dd7e87-bbf6-454b-b8e3-005051d8d426 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:13.679378 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-6e9c3c8b-e921-49e7-901d-f019a7fb1384 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:13.679981 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-6e9c3c8b-e921-49e7-901d-f019a7fb1384 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:13.681830 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell0:SELECT=4 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:08:13.687727 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6e9c3c8b-e921-49e7-901d-f019a7fb1384 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:13.688171 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6e9c3c8b-e921-49e7-901d-f019a7fb1384 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:13.688956 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6e9c3c8b-e921-49e7-901d-f019a7fb1384 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:13.689400 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6e9c3c8b-e921-49e7-901d-f019a7fb1384 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:13.747740 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-f4dd7e87-bbf6-454b-b8e3-005051d8d426 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:13.758248 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-f4dd7e87-bbf6-454b-b8e3-005051d8d426 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1723 microversion: 2.1 time: 0.218028 Jul 19 11:08:13.759264 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 291/600] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:13 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1723 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:13.860268 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-7e5884dc-0b70-4e0e-95c5-37d9d95c4312 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:13.861192 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-7e5884dc-0b70-4e0e-95c5-37d9d95c4312 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:13.865832 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-6e9c3c8b-e921-49e7-901d-f019a7fb1384 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:13.867338 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7e5884dc-0b70-4e0e-95c5-37d9d95c4312 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:13.867781 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7e5884dc-0b70-4e0e-95c5-37d9d95c4312 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:13.868819 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7e5884dc-0b70-4e0e-95c5-37d9d95c4312 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:13.869703 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7e5884dc-0b70-4e0e-95c5-37d9d95c4312 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:13.875158 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-6e9c3c8b-e921-49e7-901d-f019a7fb1384 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:13.877178 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-6e9c3c8b-e921-49e7-901d-f019a7fb1384 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:13.878540 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-6e9c3c8b-e921-49e7-901d-f019a7fb1384 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1807 microversion: 2.20 time: 0.203084 Jul 19 11:08:13.879200 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 310/601] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:08:13 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1807 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:08:14.024812 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-7e5884dc-0b70-4e0e-95c5-37d9d95c4312 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:14.032323 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-7e5884dc-0b70-4e0e-95c5-37d9d95c4312 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1358 microversion: 2.1 time: 0.176253 Jul 19 11:08:14.041057 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 292/602] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:13 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1358 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:14.777244 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-ef3ec3f1-c6ac-4f5b-944d-548859759a87 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:14.778047 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-ef3ec3f1-c6ac-4f5b-944d-548859759a87 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:14.785936 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ef3ec3f1-c6ac-4f5b-944d-548859759a87 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:14.786692 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ef3ec3f1-c6ac-4f5b-944d-548859759a87 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:14.787688 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ef3ec3f1-c6ac-4f5b-944d-548859759a87 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:14.788312 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ef3ec3f1-c6ac-4f5b-944d-548859759a87 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:14.898813 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-06e1c3cf-20a2-4c22-8da8-505676bb7742 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:14.900458 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-06e1c3cf-20a2-4c22-8da8-505676bb7742 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:14.904052 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=4 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:08:14.908158 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-06e1c3cf-20a2-4c22-8da8-505676bb7742 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:14.908680 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-06e1c3cf-20a2-4c22-8da8-505676bb7742 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:14.909478 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-06e1c3cf-20a2-4c22-8da8-505676bb7742 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:14.910167 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-06e1c3cf-20a2-4c22-8da8-505676bb7742 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:14.935460 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-ef3ec3f1-c6ac-4f5b-944d-548859759a87 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:14.945446 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-ef3ec3f1-c6ac-4f5b-944d-548859759a87 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1723 microversion: 2.1 time: 0.173704 Jul 19 11:08:14.946241 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 311/603] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:14 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1723 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:15.054589 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-06e1c3cf-20a2-4c22-8da8-505676bb7742 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:15.058411 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-5ec5301d-6a5c-4c4e-9a56-a4d0745d951b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:15.059241 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-5ec5301d-6a5c-4c4e-9a56-a4d0745d951b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:15.064222 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-06e1c3cf-20a2-4c22-8da8-505676bb7742 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:15.065735 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5ec5301d-6a5c-4c4e-9a56-a4d0745d951b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:15.065925 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-06e1c3cf-20a2-4c22-8da8-505676bb7742 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:15.066185 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5ec5301d-6a5c-4c4e-9a56-a4d0745d951b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:15.067027 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-06e1c3cf-20a2-4c22-8da8-505676bb7742 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1807 microversion: 2.20 time: 0.172976 Jul 19 11:08:15.067163 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5ec5301d-6a5c-4c4e-9a56-a4d0745d951b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:15.067580 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 293/604] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:08:14 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1807 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:08:15.067904 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5ec5301d-6a5c-4c4e-9a56-a4d0745d951b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:15.214316 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-5ec5301d-6a5c-4c4e-9a56-a4d0745d951b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:15.222170 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-5ec5301d-6a5c-4c4e-9a56-a4d0745d951b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1358 microversion: 2.1 time: 0.167623 Jul 19 11:08:15.223435 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 312/605] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:15 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1358 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:15.962891 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-052ca59f-a1a7-4ede-ad77-e941d6ea1a46 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:15.963498 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-052ca59f-a1a7-4ede-ad77-e941d6ea1a46 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:15.970060 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-052ca59f-a1a7-4ede-ad77-e941d6ea1a46 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:15.970503 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-052ca59f-a1a7-4ede-ad77-e941d6ea1a46 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:15.971266 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-052ca59f-a1a7-4ede-ad77-e941d6ea1a46 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:15.971694 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-052ca59f-a1a7-4ede-ad77-e941d6ea1a46 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:16.093627 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-e5fd53d9-194f-489c-8137-3c249721126c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:16.094236 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-e5fd53d9-194f-489c-8137-3c249721126c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:16.101487 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e5fd53d9-194f-489c-8137-3c249721126c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:16.102101 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e5fd53d9-194f-489c-8137-3c249721126c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:16.102874 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e5fd53d9-194f-489c-8137-3c249721126c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:16.103522 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e5fd53d9-194f-489c-8137-3c249721126c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:16.114611 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-052ca59f-a1a7-4ede-ad77-e941d6ea1a46 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:16.125332 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-052ca59f-a1a7-4ede-ad77-e941d6ea1a46 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1723 microversion: 2.1 time: 0.165935 Jul 19 11:08:16.126122 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 294/606] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:15 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1723 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:16.239942 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-c462b3dc-e6b7-4db2-bfea-9579b74efb22 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:16.240480 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-c462b3dc-e6b7-4db2-bfea-9579b74efb22 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:16.244468 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-e5fd53d9-194f-489c-8137-3c249721126c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:16.247258 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c462b3dc-e6b7-4db2-bfea-9579b74efb22 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:16.247703 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c462b3dc-e6b7-4db2-bfea-9579b74efb22 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:16.248981 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c462b3dc-e6b7-4db2-bfea-9579b74efb22 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:16.249382 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c462b3dc-e6b7-4db2-bfea-9579b74efb22 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:16.254338 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-e5fd53d9-194f-489c-8137-3c249721126c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:16.256010 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-e5fd53d9-194f-489c-8137-3c249721126c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:16.257244 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-e5fd53d9-194f-489c-8137-3c249721126c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1807 microversion: 2.20 time: 0.167534 Jul 19 11:08:16.259637 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 313/607] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:08:16 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1807 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:08:16.382783 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-c462b3dc-e6b7-4db2-bfea-9579b74efb22 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:16.390235 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-c462b3dc-e6b7-4db2-bfea-9579b74efb22 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1358 microversion: 2.1 time: 0.154348 Jul 19 11:08:16.390802 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 295/608] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:16 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1358 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:08:17.143726 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-3e43c2a7-d59a-458a-8f10-fd7ed2298e22 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:17.144358 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-3e43c2a7-d59a-458a-8f10-fd7ed2298e22 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:17.151353 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3e43c2a7-d59a-458a-8f10-fd7ed2298e22 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:17.151792 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3e43c2a7-d59a-458a-8f10-fd7ed2298e22 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:17.152568 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3e43c2a7-d59a-458a-8f10-fd7ed2298e22 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:17.153074 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3e43c2a7-d59a-458a-8f10-fd7ed2298e22 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:17.275573 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-986a61da-7c26-4dd0-a799-ab0c688a948b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:17.276794 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-986a61da-7c26-4dd0-a799-ab0c688a948b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:17.283213 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-3e43c2a7-d59a-458a-8f10-fd7ed2298e22 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:17.283379 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-986a61da-7c26-4dd0-a799-ab0c688a948b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:17.284007 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-986a61da-7c26-4dd0-a799-ab0c688a948b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:17.285106 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-986a61da-7c26-4dd0-a799-ab0c688a948b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:17.285651 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-986a61da-7c26-4dd0-a799-ab0c688a948b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:17.291972 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-3e43c2a7-d59a-458a-8f10-fd7ed2298e22 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1700 microversion: 2.1 time: 0.152088 Jul 19 11:08:17.292536 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 314/609] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:17 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1700 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:17.308099 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-b730e7e9-9fae-4c1b-ae40-18c7cc19ba7d tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:08:17.314212 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b730e7e9-9fae-4c1b-ae40-18c7cc19ba7d tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:17.314671 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b730e7e9-9fae-4c1b-ae40-18c7cc19ba7d tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:17.373929 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-b730e7e9-9fae-4c1b-ae40-18c7cc19ba7d tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Going to try to start instance {{(pid=117848) start /opt/stack/nova/nova/compute/api.py:2730}} Jul 19 11:08:17.419306 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-b730e7e9-9fae-4c1b-ae40-18c7cc19ba7d tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "POST /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487/action" status: 202 len: 0 microversion: 2.1 time: 0.114443 Jul 19 11:08:17.419697 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 315/610] 10.222.0.33 () {60 vars in 1277 bytes} [Tue Jul 19 11:08:17 2022] POST /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487/action => generated 0 bytes in 116 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 19 11:08:17.424920 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-8fa793cb-23ef-49a6-8019-841159ff9d94 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:17.425201 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-986a61da-7c26-4dd0-a799-ab0c688a948b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:17.425615 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-8fa793cb-23ef-49a6-8019-841159ff9d94 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:17.431832 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8fa793cb-23ef-49a6-8019-841159ff9d94 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:17.432281 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8fa793cb-23ef-49a6-8019-841159ff9d94 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:17.433088 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8fa793cb-23ef-49a6-8019-841159ff9d94 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:17.433506 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8fa793cb-23ef-49a6-8019-841159ff9d94 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:17.434277 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-986a61da-7c26-4dd0-a799-ab0c688a948b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:17.435894 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-986a61da-7c26-4dd0-a799-ab0c688a948b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:17.437159 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-986a61da-7c26-4dd0-a799-ab0c688a948b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1807 microversion: 2.20 time: 0.165909 Jul 19 11:08:17.437792 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 296/611] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:08:17 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1807 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:08:17.442197 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-ceb8a3d4-b92a-45c5-85de-0ce98a451fb8 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:17.442866 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-ceb8a3d4-b92a-45c5-85de-0ce98a451fb8 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:17.448728 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ceb8a3d4-b92a-45c5-85de-0ce98a451fb8 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:17.449200 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ceb8a3d4-b92a-45c5-85de-0ce98a451fb8 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:17.449962 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ceb8a3d4-b92a-45c5-85de-0ce98a451fb8 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:17.450395 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ceb8a3d4-b92a-45c5-85de-0ce98a451fb8 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:17.562483 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-8fa793cb-23ef-49a6-8019-841159ff9d94 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:17.570708 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-8fa793cb-23ef-49a6-8019-841159ff9d94 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1358 microversion: 2.1 time: 0.149818 Jul 19 11:08:17.571201 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 316/612] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:17 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1358 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:17.580627 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-ceb8a3d4-b92a-45c5-85de-0ce98a451fb8 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:17.589135 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-ceb8a3d4-b92a-45c5-85de-0ce98a451fb8 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1709 microversion: 2.1 time: 0.150387 Jul 19 11:08:17.589684 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 297/613] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:17 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1709 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:18.455197 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-9087388a-8705-46fa-a5c9-fb4d66273942 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:18.455868 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-9087388a-8705-46fa-a5c9-fb4d66273942 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:18.462338 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9087388a-8705-46fa-a5c9-fb4d66273942 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:18.462779 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9087388a-8705-46fa-a5c9-fb4d66273942 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:18.463535 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9087388a-8705-46fa-a5c9-fb4d66273942 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:18.463965 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9087388a-8705-46fa-a5c9-fb4d66273942 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:18.587195 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-f1975850-aed3-4cf3-a56b-4331387af189 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:18.587884 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-f1975850-aed3-4cf3-a56b-4331387af189 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:18.593936 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f1975850-aed3-4cf3-a56b-4331387af189 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:18.594382 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f1975850-aed3-4cf3-a56b-4331387af189 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:18.595134 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f1975850-aed3-4cf3-a56b-4331387af189 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:18.595562 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f1975850-aed3-4cf3-a56b-4331387af189 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:18.596006 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-9087388a-8705-46fa-a5c9-fb4d66273942 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:18.605248 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-9087388a-8705-46fa-a5c9-fb4d66273942 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:18.606851 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-9087388a-8705-46fa-a5c9-fb4d66273942 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:18.608132 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-9087388a-8705-46fa-a5c9-fb4d66273942 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1807 microversion: 2.20 time: 0.156852 Jul 19 11:08:18.608775 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 317/614] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:08:18 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1807 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:08:18.613865 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-1ff754e1-1c10-43c5-97a6-a936ac90332b tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:18.614577 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-1ff754e1-1c10-43c5-97a6-a936ac90332b tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:18.621396 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1ff754e1-1c10-43c5-97a6-a936ac90332b tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:18.621851 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1ff754e1-1c10-43c5-97a6-a936ac90332b tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:18.622621 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1ff754e1-1c10-43c5-97a6-a936ac90332b tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:18.623063 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1ff754e1-1c10-43c5-97a6-a936ac90332b tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:18.725821 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-f1975850-aed3-4cf3-a56b-4331387af189 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:18.733409 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-f1975850-aed3-4cf3-a56b-4331387af189 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1358 microversion: 2.1 time: 0.149976 Jul 19 11:08:18.733984 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 298/615] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:18 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1358 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:08:18.757224 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-1ff754e1-1c10-43c5-97a6-a936ac90332b tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:18.765056 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-1ff754e1-1c10-43c5-97a6-a936ac90332b tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1709 microversion: 2.1 time: 0.154623 Jul 19 11:08:18.765615 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 318/616] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:18 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1709 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:19.627096 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-d9246ef8-6ce9-4a59-a47b-8cf31ec50166 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:19.627697 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-d9246ef8-6ce9-4a59-a47b-8cf31ec50166 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:19.634437 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d9246ef8-6ce9-4a59-a47b-8cf31ec50166 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:19.634883 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d9246ef8-6ce9-4a59-a47b-8cf31ec50166 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:19.635637 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d9246ef8-6ce9-4a59-a47b-8cf31ec50166 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:19.636069 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d9246ef8-6ce9-4a59-a47b-8cf31ec50166 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:19.750577 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-f7e78333-a74c-4a85-9e3d-be6e8cdddb25 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:19.751106 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-f7e78333-a74c-4a85-9e3d-be6e8cdddb25 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:19.758151 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f7e78333-a74c-4a85-9e3d-be6e8cdddb25 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:19.758597 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f7e78333-a74c-4a85-9e3d-be6e8cdddb25 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:19.759357 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f7e78333-a74c-4a85-9e3d-be6e8cdddb25 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:19.759786 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f7e78333-a74c-4a85-9e3d-be6e8cdddb25 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:19.767733 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-d9246ef8-6ce9-4a59-a47b-8cf31ec50166 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:19.776158 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-d9246ef8-6ce9-4a59-a47b-8cf31ec50166 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:19.777777 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-d9246ef8-6ce9-4a59-a47b-8cf31ec50166 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:19.778927 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-d9246ef8-6ce9-4a59-a47b-8cf31ec50166 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1807 microversion: 2.20 time: 0.155987 Jul 19 11:08:19.779648 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 299/617] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:08:19 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1807 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:08:19.783947 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-18a921cd-56a0-4e95-a428-34afdc9f805c tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:19.784543 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-18a921cd-56a0-4e95-a428-34afdc9f805c tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:19.793829 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-18a921cd-56a0-4e95-a428-34afdc9f805c tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:19.794293 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-18a921cd-56a0-4e95-a428-34afdc9f805c tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:19.795111 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-18a921cd-56a0-4e95-a428-34afdc9f805c tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:19.795553 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-18a921cd-56a0-4e95-a428-34afdc9f805c tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:19.888774 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-f7e78333-a74c-4a85-9e3d-be6e8cdddb25 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:19.896423 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-f7e78333-a74c-4a85-9e3d-be6e8cdddb25 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1548 microversion: 2.1 time: 0.150047 Jul 19 11:08:19.897096 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 319/618] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:19 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1548 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:08:19.912212 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-4ea69380-ed4a-482e-921e-9d9a257602a4 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:19.913416 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-4ea69380-ed4a-482e-921e-9d9a257602a4 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:19.919710 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4ea69380-ed4a-482e-921e-9d9a257602a4 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:19.920348 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4ea69380-ed4a-482e-921e-9d9a257602a4 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:19.921366 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4ea69380-ed4a-482e-921e-9d9a257602a4 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:19.921804 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4ea69380-ed4a-482e-921e-9d9a257602a4 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:19.927229 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-18a921cd-56a0-4e95-a428-34afdc9f805c tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:19.936768 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-18a921cd-56a0-4e95-a428-34afdc9f805c tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1709 microversion: 2.1 time: 0.156208 Jul 19 11:08:19.937355 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 300/619] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:19 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1709 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:20.054690 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-4ea69380-ed4a-482e-921e-9d9a257602a4 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:20.062071 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-4ea69380-ed4a-482e-921e-9d9a257602a4 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1548 microversion: 2.1 time: 0.153551 Jul 19 11:08:20.062628 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 320/620] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:19 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1548 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:20.077571 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-3f017e10-ecbc-4f1f-9d64-dfe4c6be0072 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:20.081038 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-3f017e10-ecbc-4f1f-9d64-dfe4c6be0072 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:20.081826 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-3f017e10-ecbc-4f1f-9d64-dfe4c6be0072 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Searching by: {'deleted': False, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a'} {{(pid=117849) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 19 11:08:20.088668 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3f017e10-ecbc-4f1f-9d64-dfe4c6be0072 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:20.089177 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3f017e10-ecbc-4f1f-9d64-dfe4c6be0072 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:20.089964 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3f017e10-ecbc-4f1f-9d64-dfe4c6be0072 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:20.090386 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3f017e10-ecbc-4f1f-9d64-dfe4c6be0072 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:20.180722 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.multi_cell_list [None req-3f017e10-ecbc-4f1f-9d64-dfe4c6be0072 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=117849) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 19 11:08:20.273447 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-3f017e10-ecbc-4f1f-9d64-dfe4c6be0072 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers" status: 200 len: 343 microversion: 2.1 time: 0.199448 Jul 19 11:08:20.273931 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 301/621] 10.222.0.33 () {58 vars in 1124 bytes} [Tue Jul 19 11:08:20 2022] GET /compute/v2.1/servers => generated 343 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:08:20.284119 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-e323e5ab-73af-429b-a15d-a43e520b063d tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:20.284741 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-e323e5ab-73af-429b-a15d-a43e520b063d tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:20.291345 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e323e5ab-73af-429b-a15d-a43e520b063d tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:20.291787 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e323e5ab-73af-429b-a15d-a43e520b063d tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:20.292532 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e323e5ab-73af-429b-a15d-a43e520b063d tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:20.292995 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e323e5ab-73af-429b-a15d-a43e520b063d tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:20.417860 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-e323e5ab-73af-429b-a15d-a43e520b063d tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:20.425145 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-e323e5ab-73af-429b-a15d-a43e520b063d tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1548 microversion: 2.1 time: 0.144694 Jul 19 11:08:20.425706 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 321/622] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:20 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1548 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:20.798311 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-72840e49-1bd7-4192-929d-20270509f526 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:20.799124 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-72840e49-1bd7-4192-929d-20270509f526 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:20.805107 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-72840e49-1bd7-4192-929d-20270509f526 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:20.805561 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-72840e49-1bd7-4192-929d-20270509f526 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:20.806324 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-72840e49-1bd7-4192-929d-20270509f526 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:20.806760 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-72840e49-1bd7-4192-929d-20270509f526 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:20.934824 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-72840e49-1bd7-4192-929d-20270509f526 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:20.948964 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-72840e49-1bd7-4192-929d-20270509f526 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:20.950393 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-72840e49-1bd7-4192-929d-20270509f526 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:20.951804 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-72840e49-1bd7-4192-929d-20270509f526 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1807 microversion: 2.20 time: 0.157695 Jul 19 11:08:20.952538 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 302/623] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:08:20 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1807 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:08:20.954353 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-eede839d-a84a-4021-ad09-a467ef20baee tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:20.954966 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-eede839d-a84a-4021-ad09-a467ef20baee tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:20.963232 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-eede839d-a84a-4021-ad09-a467ef20baee tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:20.963702 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-eede839d-a84a-4021-ad09-a467ef20baee tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:20.964473 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-eede839d-a84a-4021-ad09-a467ef20baee tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:20.964940 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-eede839d-a84a-4021-ad09-a467ef20baee tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:21.110066 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-eede839d-a84a-4021-ad09-a467ef20baee tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:21.113841 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:08:21.119530 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-eede839d-a84a-4021-ad09-a467ef20baee tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1709 microversion: 2.1 time: 0.169048 Jul 19 11:08:21.120115 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 322/624] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:20 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1709 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:21.944950 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-719e4dbf-c723-485f-9564-f4bcf220f667 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "4be186cd-8f4e-4623-ba8c-2bc3eb5565b6"}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:08:21.946791 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-719e4dbf-c723-485f-9564-f4bcf220f667 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:21.953096 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-719e4dbf-c723-485f-9564-f4bcf220f667 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:21.953543 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-719e4dbf-c723-485f-9564-f4bcf220f667 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:21.954303 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-719e4dbf-c723-485f-9564-f4bcf220f667 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:21.954733 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-719e4dbf-c723-485f-9564-f4bcf220f667 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:21.970340 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-0153dd66-20bd-41a2-baf7-4b55a9d2002f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:21.971045 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-0153dd66-20bd-41a2-baf7-4b55a9d2002f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:21.979257 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0153dd66-20bd-41a2-baf7-4b55a9d2002f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:21.979642 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0153dd66-20bd-41a2-baf7-4b55a9d2002f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:21.980421 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0153dd66-20bd-41a2-baf7-4b55a9d2002f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:21.980898 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0153dd66-20bd-41a2-baf7-4b55a9d2002f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:22.120650 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-0153dd66-20bd-41a2-baf7-4b55a9d2002f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:22.133684 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-0153dd66-20bd-41a2-baf7-4b55a9d2002f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:22.135279 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-0153dd66-20bd-41a2-baf7-4b55a9d2002f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:22.136690 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-0153dd66-20bd-41a2-baf7-4b55a9d2002f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1807 microversion: 2.20 time: 0.170642 Jul 19 11:08:22.137730 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 323/625] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:08:21 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1807 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:08:22.142176 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-428b6f5a-dcde-492c-8db6-1bcb021b3c99 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:22.142774 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-428b6f5a-dcde-492c-8db6-1bcb021b3c99 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:22.150427 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-428b6f5a-dcde-492c-8db6-1bcb021b3c99 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:22.150957 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-428b6f5a-dcde-492c-8db6-1bcb021b3c99 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:22.151908 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-428b6f5a-dcde-492c-8db6-1bcb021b3c99 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:22.152506 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-428b6f5a-dcde-492c-8db6-1bcb021b3c99 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:22.276744 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-719e4dbf-c723-485f-9564-f4bcf220f667 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "POST /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.335718 Jul 19 11:08:22.277649 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 303/626] 10.222.0.33 () {60 vars in 1322 bytes} [Tue Jul 19 11:08:21 2022] POST /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a/os-volume_attachments => generated 194 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:08:22.283114 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-e6e8b26d-ff5b-41b8-8e1f-c6f03bf69f55 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-800804149", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1205270003", "security_groups": [{"name": "tempest-secgroup-smoke-1334719817"}], "block_device_mapping_v2": [{"uuid": "2125ea25-7fd0-42f0-ace7-b5ad9ed45ebd", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "1660664f-e111-4b9d-93da-a8c613b3e4df"}]}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:08:22.310076 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-428b6f5a-dcde-492c-8db6-1bcb021b3c99 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:22.319670 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-428b6f5a-dcde-492c-8db6-1bcb021b3c99 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1713 microversion: 2.1 time: 0.180414 Jul 19 11:08:22.329246 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 324/627] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:22 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1713 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:22.412509 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.network.neutron [None req-e6e8b26d-ff5b-41b8-8e1f-c6f03bf69f55 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] validate_networks() for [('1660664f-e111-4b9d-93da-a8c613b3e4df', None, None, None, None, None)] {{(pid=117849) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 19 11:08:22.776326 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-e6e8b26d-ff5b-41b8-8e1f-c6f03bf69f55 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Flavor limits 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 19 11:08:22.776785 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-e6e8b26d-ff5b-41b8-8e1f-c6f03bf69f55 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Image limits 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 19 11:08:22.777292 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-e6e8b26d-ff5b-41b8-8e1f-c6f03bf69f55 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Flavor pref 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 19 11:08:22.777697 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-e6e8b26d-ff5b-41b8-8e1f-c6f03bf69f55 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Image pref 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 19 11:08:22.778102 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-e6e8b26d-ff5b-41b8-8e1f-c6f03bf69f55 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 19 11:08:22.786806 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e6e8b26d-ff5b-41b8-8e1f-c6f03bf69f55 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:22.787251 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e6e8b26d-ff5b-41b8-8e1f-c6f03bf69f55 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:22.788586 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e6e8b26d-ff5b-41b8-8e1f-c6f03bf69f55 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:22.789051 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e6e8b26d-ff5b-41b8-8e1f-c6f03bf69f55 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:22.806187 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-e6e8b26d-ff5b-41b8-8e1f-c6f03bf69f55 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Getting quotas for project a7109adf88ac4391aee35c150e96e0b1. Resources: {'cores', 'instances', 'ram'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:08:22.810345 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-e6e8b26d-ff5b-41b8-8e1f-c6f03bf69f55 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Getting quotas for user a217e983effa43f3a4f84221984189c5 and project a7109adf88ac4391aee35c150e96e0b1. Resources: {'cores', 'instances', 'ram'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:08:22.845654 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-f8da5d74-da33-4d7d-9fba-e680638198bf tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:22.846142 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-f8da5d74-da33-4d7d-9fba-e680638198bf tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 988f14d7-6354-4388-a51c-b4df9749c671] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:22.852751 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f8da5d74-da33-4d7d-9fba-e680638198bf tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:22.853260 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f8da5d74-da33-4d7d-9fba-e680638198bf tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:22.854014 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f8da5d74-da33-4d7d-9fba-e680638198bf tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:22.854445 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f8da5d74-da33-4d7d-9fba-e680638198bf tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:22.866980 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-e6e8b26d-ff5b-41b8-8e1f-c6f03bf69f55 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Going to run 1 instances... {{(pid=117849) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 19 11:08:22.869198 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-e6e8b26d-ff5b-41b8-8e1f-c6f03bf69f55 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 12cd8c55-b30f-40ee-8cf5-7c974b8caafb] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='2125ea25-7fd0-42f0-ace7-b5ad9ed45ebd',volume_size=None,volume_type=None)] {{(pid=117849) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 19 11:08:22.877870 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-f8da5d74-da33-4d7d-9fba-e680638198bf tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671/ips" status: 200 len: 151 microversion: 2.1 time: 0.035988 Jul 19 11:08:22.883761 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 325/628] 10.222.0.33 () {58 vars in 1247 bytes} [Tue Jul 19 11:08:22 2022] GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671/ips => generated 151 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:08:22.888739 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-1683df79-0c13-4bca-b28e-815a04075a34 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "45d056f5-25af-475a-8cf5-ff31a13487c2"}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:08:22.889878 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-1683df79-0c13-4bca-b28e-815a04075a34 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 988f14d7-6354-4388-a51c-b4df9749c671] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:22.896044 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1683df79-0c13-4bca-b28e-815a04075a34 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:22.896493 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1683df79-0c13-4bca-b28e-815a04075a34 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:22.897332 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1683df79-0c13-4bca-b28e-815a04075a34 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:22.897764 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1683df79-0c13-4bca-b28e-815a04075a34 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:22.996355 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-e6e8b26d-ff5b-41b8-8e1f-c6f03bf69f55 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.717229 Jul 19 11:08:22.997109 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 304/629] 10.222.0.33 () {60 vars in 1146 bytes} [Tue Jul 19 11:08:22 2022] POST /compute/v2.1/servers => generated 406 bytes in 719 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 19 11:08:22.999803 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.wsgi [None req-1683df79-0c13-4bca-b28e-815a04075a34 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] HTTP exception thrown: Invalid volume: volume 45d056f5-25af-475a-8cf5-ff31a13487c2 is already attached to instances: 4c71f564-de63-4fcd-88ce-01ecba3d8372 Jul 19 11:08:23.001547 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-1683df79-0c13-4bca-b28e-815a04075a34 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Request method failure captured: Jul 19 11:08:23.001547 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: request: POST /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671/os-volume_attachments HTTP/1.1 Jul 19 11:08:23.001547 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept: application/json Jul 19 11:08:23.001547 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept-Encoding: identity Jul 19 11:08:23.001547 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Connection: close Jul 19 11:08:23.001547 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Length: 74 Jul 19 11:08:23.001547 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Type: application/json Jul 19 11:08:23.001547 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Host: 10.222.0.33 Jul 19 11:08:23.001547 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Openstack-System-Scope: None Jul 19 11:08:23.001547 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: User-Agent: python-urllib3/1.26.9 Jul 19 11:08:23.001547 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Auth-Token: gAAAAABi1o_HYpL7r_umyWTTKsnaQkfxgRQPrCVzXYV3YxPvpHKAUsKkXxPNkPyHBhEDbahQjtwakXlDpqARj9YKw3tFYmG0gCU4SGJZDCiqyWHloRDABrhK0mtm6JR4S49vBX-Zeg1oz_jYGWaYn6kU0GfIXikwtQO5JQ3VMjNcjqJ15uYlE9Q Jul 19 11:08:23.001547 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Id: None Jul 19 11:08:23.001547 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Name: None Jul 19 11:08:23.001547 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Identity-Status: Confirmed Jul 19 11:08:23.001547 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Is-Admin-Project: True Jul 19 11:08:23.001547 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Id: default Jul 19 11:08:23.001547 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Name: Default Jul 19 11:08:23.001547 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Id: ef534f1d06ea415d98857ec7f9e5fb97 Jul 19 11:08:23.001547 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Name: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:08:23.001547 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Role: member,reader Jul 19 11:08:23.001547 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Roles: member,reader Jul 19 11:08:23.002961 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:08:23.002961 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:08:23.002961 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Id: ef534f1d06ea415d98857ec7f9e5fb97 Jul 19 11:08:23.002961 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:08:23.002961 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User: tempest-AttachVolumeNegativeTest-434278602-project Jul 19 11:08:23.002961 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Id: default Jul 19 11:08:23.002961 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Name: Default Jul 19 11:08:23.002961 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Id: b8f2a78f3a3e4b8e8937e3bc39524690 Jul 19 11:08:23.002961 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Name: tempest-AttachVolumeNegativeTest-434278602-project Jul 19 11:08:23.002961 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Jul 19 11:08:23.002961 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: {"volumeAttachment": {"volumeId": "45d056f5-25af-475a-8cf5-ff31a13487c2"}} Jul 19 11:08:23.002961 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: method: > Jul 19 11:08:23.003859 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: exception: Invalid volume: volume 45d056f5-25af-475a-8cf5-ff31a13487c2 is already attached to instances: 4c71f564-de63-4fcd-88ce-01ecba3d8372 Jul 19 11:08:23.003859 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: action_args: {'server_id': '988f14d7-6354-4388-a51c-b4df9749c671', 'body': {'volumeAttachment': {'volumeId': '45d056f5-25af-475a-8cf5-ff31a13487c2'}}} Jul 19 11:08:23.003859 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:08:23.003859 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:08:23.003859 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Jul 19 11:08:23.003859 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Jul 19 11:08:23.003859 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 19 11:08:23.003859 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 19 11:08:23.003859 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 19 11:08:23.003859 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 19 11:08:23.003859 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4985, in attach_volume Jul 19 11:08:23.003859 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Jul 19 11:08:23.003859 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4825, in _check_volume_already_attached Jul 19 11:08:23.003859 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Jul 19 11:08:23.003859 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 45d056f5-25af-475a-8cf5-ff31a13487c2 is already attached to instances: 4c71f564-de63-4fcd-88ce-01ecba3d8372 Jul 19 11:08:23.003859 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:08:23.003859 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:08:23.003859 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:08:23.003859 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:08:23.005753 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:08:23.005753 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:08:23.005753 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:08:23.005753 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:08:23.005753 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:08:23.005753 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:08:23.005753 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 19 11:08:23.005753 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 19 11:08:23.005753 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 19 11:08:23.005753 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 19 11:08:23.005753 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 19 11:08:23.005753 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 19 11:08:23.005753 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Jul 19 11:08:23.005753 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 19 11:08:23.005753 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 45d056f5-25af-475a-8cf5-ff31a13487c2 is already attached to instances: 4c71f564-de63-4fcd-88ce-01ecba3d8372 Jul 19 11:08:23.005753 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:08:23.005753 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:08:23.005753 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:08:23.005753 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:08:23.005753 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:08:23.008267 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:08:23.008267 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:08:23.008267 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:08:23.008267 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 45d056f5-25af-475a-8cf5-ff31a13487c2 is already attached to instances: 4c71f564-de63-4fcd-88ce-01ecba3d8372 Jul 19 11:08:23.008267 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:08:23.008267 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-1683df79-0c13-4bca-b28e-815a04075a34 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Returning 400 to user: Invalid volume: volume 45d056f5-25af-475a-8cf5-ff31a13487c2 is already attached to instances: 4c71f564-de63-4fcd-88ce-01ecba3d8372 {{(pid=117848) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:08:23.008267 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-1683df79-0c13-4bca-b28e-815a04075a34 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "POST /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.118143 Jul 19 11:08:23.008267 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 326/630] 10.222.0.33 () {60 vars in 1322 bytes} [Tue Jul 19 11:08:22 2022] POST /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671/os-volume_attachments => generated 174 bytes in 119 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Jul 19 11:08:23.010670 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-c14b2f4b-4f0c-488f-b9de-27a7dac3ebd6 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:23.011349 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-c14b2f4b-4f0c-488f-b9de-27a7dac3ebd6 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 12cd8c55-b30f-40ee-8cf5-7c974b8caafb] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:23.011503 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-14d799d5-13fc-415b-b14e-009e9d45e585 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:23.011933 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-14d799d5-13fc-415b-b14e-009e9d45e585 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 988f14d7-6354-4388-a51c-b4df9749c671] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:23.030456 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-14d799d5-13fc-415b-b14e-009e9d45e585 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:23.030914 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-14d799d5-13fc-415b-b14e-009e9d45e585 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:23.031690 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-14d799d5-13fc-415b-b14e-009e9d45e585 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:23.032128 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-14d799d5-13fc-415b-b14e-009e9d45e585 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:23.058341 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-14d799d5-13fc-415b-b14e-009e9d45e585 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 988f14d7-6354-4388-a51c-b4df9749c671] Going to try to terminate instance {{(pid=117848) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 19 11:08:23.063721 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-c14b2f4b-4f0c-488f-b9de-27a7dac3ebd6 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:23.070466 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-c14b2f4b-4f0c-488f-b9de-27a7dac3ebd6 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb" status: 200 len: 1094 microversion: 2.1 time: 0.066270 Jul 19 11:08:23.071195 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 305/631] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:23 2022] GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb => generated 1094 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:23.155431 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-2b3bacb4-50a6-4335-968f-164f9f6db807 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:23.156131 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-2b3bacb4-50a6-4335-968f-164f9f6db807 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:23.156268 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-14d799d5-13fc-415b-b14e-009e9d45e585 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "DELETE /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671" status: 204 len: 0 microversion: 2.1 time: 0.148095 Jul 19 11:08:23.157181 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 327/632] 10.222.0.33 () {58 vars in 1238 bytes} [Tue Jul 19 11:08:23 2022] DELETE /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 19 11:08:23.163254 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2b3bacb4-50a6-4335-968f-164f9f6db807 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:23.163791 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2b3bacb4-50a6-4335-968f-164f9f6db807 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:23.164796 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2b3bacb4-50a6-4335-968f-164f9f6db807 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:23.165413 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2b3bacb4-50a6-4335-968f-164f9f6db807 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:23.167128 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-5bdfb542-beee-4a0d-b51a-9384ffdd8e13 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:23.167953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-5bdfb542-beee-4a0d-b51a-9384ffdd8e13 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 988f14d7-6354-4388-a51c-b4df9749c671] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:23.171005 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:08:23.175317 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5bdfb542-beee-4a0d-b51a-9384ffdd8e13 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:23.175868 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5bdfb542-beee-4a0d-b51a-9384ffdd8e13 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:23.176988 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5bdfb542-beee-4a0d-b51a-9384ffdd8e13 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:23.177503 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5bdfb542-beee-4a0d-b51a-9384ffdd8e13 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:23.315681 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-2b3bacb4-50a6-4335-968f-164f9f6db807 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:23.325191 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-2b3bacb4-50a6-4335-968f-164f9f6db807 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:23.327221 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-2b3bacb4-50a6-4335-968f-164f9f6db807 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:23.328800 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-2b3bacb4-50a6-4335-968f-164f9f6db807 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1807 microversion: 2.20 time: 0.177159 Jul 19 11:08:23.329584 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 306/633] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:08:23 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1807 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:08:23.339420 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-5bdfb542-beee-4a0d-b51a-9384ffdd8e13 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:23.346352 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-5bdfb542-beee-4a0d-b51a-9384ffdd8e13 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671" status: 200 len: 1680 microversion: 2.1 time: 0.184765 Jul 19 11:08:23.346912 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 328/634] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:23 2022] GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671 => generated 1680 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:24.086470 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-19c4b89a-792f-4ae8-baba-657c0a98c371 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:24.086983 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-19c4b89a-792f-4ae8-baba-657c0a98c371 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 12cd8c55-b30f-40ee-8cf5-7c974b8caafb] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:24.093363 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-19c4b89a-792f-4ae8-baba-657c0a98c371 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:24.093822 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-19c4b89a-792f-4ae8-baba-657c0a98c371 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:24.094689 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-19c4b89a-792f-4ae8-baba-657c0a98c371 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:24.095212 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-19c4b89a-792f-4ae8-baba-657c0a98c371 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:24.160077 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-19c4b89a-792f-4ae8-baba-657c0a98c371 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:24.167430 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-19c4b89a-792f-4ae8-baba-657c0a98c371 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb" status: 200 len: 1144 microversion: 2.1 time: 0.084991 Jul 19 11:08:24.167993 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 307/635] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:24 2022] GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb => generated 1144 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:24.347053 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-64731914-cdcb-4e9e-9245-c8a5bc98762d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:24.347580 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-64731914-cdcb-4e9e-9245-c8a5bc98762d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:24.354763 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-64731914-cdcb-4e9e-9245-c8a5bc98762d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:24.355212 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-64731914-cdcb-4e9e-9245-c8a5bc98762d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:24.355968 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-64731914-cdcb-4e9e-9245-c8a5bc98762d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:24.356404 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-64731914-cdcb-4e9e-9245-c8a5bc98762d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:24.364527 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-6fae47c1-1a08-4b26-8767-d4f6977cd338 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:24.365244 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-6fae47c1-1a08-4b26-8767-d4f6977cd338 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 988f14d7-6354-4388-a51c-b4df9749c671] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:24.372251 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6fae47c1-1a08-4b26-8767-d4f6977cd338 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:24.372728 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6fae47c1-1a08-4b26-8767-d4f6977cd338 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:24.373754 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6fae47c1-1a08-4b26-8767-d4f6977cd338 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:24.374190 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6fae47c1-1a08-4b26-8767-d4f6977cd338 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:24.488373 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-64731914-cdcb-4e9e-9245-c8a5bc98762d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:24.497543 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-64731914-cdcb-4e9e-9245-c8a5bc98762d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:24.499314 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-64731914-cdcb-4e9e-9245-c8a5bc98762d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:24.500455 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-64731914-cdcb-4e9e-9245-c8a5bc98762d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1807 microversion: 2.20 time: 0.157429 Jul 19 11:08:24.501050 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 329/636] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:08:24 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1807 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:08:24.505994 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-6fae47c1-1a08-4b26-8767-d4f6977cd338 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:24.513229 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-6fae47c1-1a08-4b26-8767-d4f6977cd338 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671" status: 200 len: 1680 microversion: 2.1 time: 0.152555 Jul 19 11:08:24.513771 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 308/637] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:24 2022] GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671 => generated 1680 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:08:25.183935 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-a01837b6-d5de-4ea1-a11b-8552c7858c8d tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:25.184533 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-a01837b6-d5de-4ea1-a11b-8552c7858c8d tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 12cd8c55-b30f-40ee-8cf5-7c974b8caafb] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:25.190682 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a01837b6-d5de-4ea1-a11b-8552c7858c8d tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:25.191076 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a01837b6-d5de-4ea1-a11b-8552c7858c8d tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:25.191828 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a01837b6-d5de-4ea1-a11b-8552c7858c8d tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:25.192262 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a01837b6-d5de-4ea1-a11b-8552c7858c8d tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:25.251170 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-a01837b6-d5de-4ea1-a11b-8552c7858c8d tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:25.258846 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-a01837b6-d5de-4ea1-a11b-8552c7858c8d tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb" status: 200 len: 1144 microversion: 2.1 time: 0.078799 Jul 19 11:08:25.259401 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 330/638] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:25 2022] GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb => generated 1144 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:25.518332 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-dbda1860-9ab4-46fb-acc0-2e4c97d2dc14 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:25.518941 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-dbda1860-9ab4-46fb-acc0-2e4c97d2dc14 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:25.527894 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dbda1860-9ab4-46fb-acc0-2e4c97d2dc14 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:25.528735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dbda1860-9ab4-46fb-acc0-2e4c97d2dc14 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:25.529963 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dbda1860-9ab4-46fb-acc0-2e4c97d2dc14 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:25.530617 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dbda1860-9ab4-46fb-acc0-2e4c97d2dc14 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:25.540450 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-514f3d96-1a4d-4e09-827b-cd11bcedb7a9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:25.540450 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-514f3d96-1a4d-4e09-827b-cd11bcedb7a9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 988f14d7-6354-4388-a51c-b4df9749c671] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:25.552016 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-514f3d96-1a4d-4e09-827b-cd11bcedb7a9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:25.552518 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-514f3d96-1a4d-4e09-827b-cd11bcedb7a9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:25.553974 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-514f3d96-1a4d-4e09-827b-cd11bcedb7a9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:25.554470 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-514f3d96-1a4d-4e09-827b-cd11bcedb7a9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:25.682662 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-dbda1860-9ab4-46fb-acc0-2e4c97d2dc14 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:25.691733 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-dbda1860-9ab4-46fb-acc0-2e4c97d2dc14 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:25.693420 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-dbda1860-9ab4-46fb-acc0-2e4c97d2dc14 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:25.695316 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-dbda1860-9ab4-46fb-acc0-2e4c97d2dc14 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1807 microversion: 2.20 time: 0.180835 Jul 19 11:08:25.696150 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 309/639] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:08:25 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1807 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:08:25.706134 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-514f3d96-1a4d-4e09-827b-cd11bcedb7a9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:25.713679 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-514f3d96-1a4d-4e09-827b-cd11bcedb7a9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671" status: 200 len: 1680 microversion: 2.1 time: 0.186197 Jul 19 11:08:25.714241 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 331/640] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:25 2022] GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671 => generated 1680 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:26.276050 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-fc93d3c7-dbe4-4461-b377-da4b734b321e tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:26.276806 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-fc93d3c7-dbe4-4461-b377-da4b734b321e tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 12cd8c55-b30f-40ee-8cf5-7c974b8caafb] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:26.282935 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fc93d3c7-dbe4-4461-b377-da4b734b321e tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:26.283314 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fc93d3c7-dbe4-4461-b377-da4b734b321e tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:26.284064 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fc93d3c7-dbe4-4461-b377-da4b734b321e tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:26.284491 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fc93d3c7-dbe4-4461-b377-da4b734b321e tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:26.361026 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-fc93d3c7-dbe4-4461-b377-da4b734b321e tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:26.368584 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-fc93d3c7-dbe4-4461-b377-da4b734b321e tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb" status: 200 len: 1192 microversion: 2.1 time: 0.096819 Jul 19 11:08:26.369179 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 310/641] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:26 2022] GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb => generated 1192 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:26.714716 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-8f508d43-3277-4787-b984-ebd11c67c637 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:26.715331 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-8f508d43-3277-4787-b984-ebd11c67c637 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:26.722471 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8f508d43-3277-4787-b984-ebd11c67c637 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:26.722931 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8f508d43-3277-4787-b984-ebd11c67c637 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:26.724995 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8f508d43-3277-4787-b984-ebd11c67c637 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:26.724995 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8f508d43-3277-4787-b984-ebd11c67c637 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:26.733285 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-3eedd5b6-686a-4843-b0d6-0f906b5d3b78 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:26.734601 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-3eedd5b6-686a-4843-b0d6-0f906b5d3b78 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 988f14d7-6354-4388-a51c-b4df9749c671] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:26.743057 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3eedd5b6-686a-4843-b0d6-0f906b5d3b78 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:26.743928 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3eedd5b6-686a-4843-b0d6-0f906b5d3b78 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:26.745001 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3eedd5b6-686a-4843-b0d6-0f906b5d3b78 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:26.745640 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3eedd5b6-686a-4843-b0d6-0f906b5d3b78 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:26.971044 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-3eedd5b6-686a-4843-b0d6-0f906b5d3b78 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:26.978859 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-3eedd5b6-686a-4843-b0d6-0f906b5d3b78 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671" status: 200 len: 1680 microversion: 2.1 time: 0.252385 Jul 19 11:08:26.979553 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 311/642] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:26 2022] GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671 => generated 1680 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:27.106331 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-8f508d43-3277-4787-b984-ebd11c67c637 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:27.116075 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-8f508d43-3277-4787-b984-ebd11c67c637 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:27.117980 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-8f508d43-3277-4787-b984-ebd11c67c637 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:27.119539 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-8f508d43-3277-4787-b984-ebd11c67c637 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1807 microversion: 2.20 time: 0.408743 Jul 19 11:08:27.132302 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 332/643] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:08:26 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1807 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:08:27.324727 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-e5aa678c-a07c-46d7-a6aa-119b092e60f7 req-a5ee5ad9-e6c3-4343-833a-69b06424ee17 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "988f14d7-6354-4388-a51c-b4df9749c671", "name": "network-vif-deleted", "tag": "ebf84be3-6be9-46a9-b6fd-10aca9db5a6f"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:08:27.339390 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-e5aa678c-a07c-46d7-a6aa-119b092e60f7 req-a5ee5ad9-e6c3-4343-833a-69b06424ee17 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:27.339878 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-e5aa678c-a07c-46d7-a6aa-119b092e60f7 req-a5ee5ad9-e6c3-4343-833a-69b06424ee17 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:27.353722 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-e5aa678c-a07c-46d7-a6aa-119b092e60f7 req-a5ee5ad9-e6c3-4343-833a-69b06424ee17 service nova] Creating event network-vif-deleted:ebf84be3-6be9-46a9-b6fd-10aca9db5a6f for instance 988f14d7-6354-4388-a51c-b4df9749c671 on hv-cinder-maste Jul 19 11:08:27.360396 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-e5aa678c-a07c-46d7-a6aa-119b092e60f7 req-a5ee5ad9-e6c3-4343-833a-69b06424ee17 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039159 Jul 19 11:08:27.361543 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 312/644] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:08:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:08:27.386975 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-df421cf3-beaf-40a7-b953-bfbeedcbeb28 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:27.387717 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-df421cf3-beaf-40a7-b953-bfbeedcbeb28 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 12cd8c55-b30f-40ee-8cf5-7c974b8caafb] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:27.393997 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-df421cf3-beaf-40a7-b953-bfbeedcbeb28 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:27.394744 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-df421cf3-beaf-40a7-b953-bfbeedcbeb28 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:27.395612 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-df421cf3-beaf-40a7-b953-bfbeedcbeb28 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:27.396325 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-df421cf3-beaf-40a7-b953-bfbeedcbeb28 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:27.481971 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-df421cf3-beaf-40a7-b953-bfbeedcbeb28 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:27.496537 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-df421cf3-beaf-40a7-b953-bfbeedcbeb28 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb" status: 200 len: 1210 microversion: 2.1 time: 0.115335 Jul 19 11:08:27.498281 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 333/645] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:27 2022] GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb => generated 1210 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:27.996610 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-14389566-f3aa-41d7-b574-99fd4fc39c47 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:27.997333 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-14389566-f3aa-41d7-b574-99fd4fc39c47 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 988f14d7-6354-4388-a51c-b4df9749c671] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:28.004981 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-14389566-f3aa-41d7-b574-99fd4fc39c47 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:28.005427 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-14389566-f3aa-41d7-b574-99fd4fc39c47 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:28.012245 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-14389566-f3aa-41d7-b574-99fd4fc39c47 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:28.012746 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-14389566-f3aa-41d7-b574-99fd4fc39c47 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:28.044379 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.wsgi [None req-14389566-f3aa-41d7-b574-99fd4fc39c47 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] HTTP exception thrown: Instance 988f14d7-6354-4388-a51c-b4df9749c671 could not be found. Jul 19 11:08:28.049916 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-14389566-f3aa-41d7-b574-99fd4fc39c47 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Request method failure captured: Jul 19 11:08:28.049916 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: request: GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671 HTTP/1.1 Jul 19 11:08:28.049916 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept: application/json Jul 19 11:08:28.049916 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept-Encoding: identity Jul 19 11:08:28.049916 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Connection: close Jul 19 11:08:28.049916 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Length: 0 Jul 19 11:08:28.049916 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Type: application/json Jul 19 11:08:28.049916 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Host: 10.222.0.33 Jul 19 11:08:28.049916 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Openstack-System-Scope: None Jul 19 11:08:28.049916 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: User-Agent: python-urllib3/1.26.9 Jul 19 11:08:28.049916 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Auth-Token: gAAAAABi1o_HYpL7r_umyWTTKsnaQkfxgRQPrCVzXYV3YxPvpHKAUsKkXxPNkPyHBhEDbahQjtwakXlDpqARj9YKw3tFYmG0gCU4SGJZDCiqyWHloRDABrhK0mtm6JR4S49vBX-Zeg1oz_jYGWaYn6kU0GfIXikwtQO5JQ3VMjNcjqJ15uYlE9Q Jul 19 11:08:28.049916 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Id: None Jul 19 11:08:28.049916 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Name: None Jul 19 11:08:28.049916 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Identity-Status: Confirmed Jul 19 11:08:28.049916 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Is-Admin-Project: True Jul 19 11:08:28.049916 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Id: default Jul 19 11:08:28.049916 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Name: Default Jul 19 11:08:28.049916 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Id: ef534f1d06ea415d98857ec7f9e5fb97 Jul 19 11:08:28.049916 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Name: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:08:28.049916 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Role: member,reader Jul 19 11:08:28.049916 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Roles: member,reader Jul 19 11:08:28.051685 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:08:28.051685 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:08:28.051685 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Id: ef534f1d06ea415d98857ec7f9e5fb97 Jul 19 11:08:28.051685 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:08:28.051685 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User: tempest-AttachVolumeNegativeTest-434278602-project Jul 19 11:08:28.051685 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Id: default Jul 19 11:08:28.051685 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Name: Default Jul 19 11:08:28.051685 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Id: b8f2a78f3a3e4b8e8937e3bc39524690 Jul 19 11:08:28.051685 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Name: tempest-AttachVolumeNegativeTest-434278602-project Jul 19 11:08:28.051685 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: method: > Jul 19 11:08:28.051685 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: exception: Instance 988f14d7-6354-4388-a51c-b4df9749c671 could not be found. Jul 19 11:08:28.051685 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: action_args: {'id': '988f14d7-6354-4388-a51c-b4df9749c671'} Jul 19 11:08:28.052545 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:08:28.052545 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:08:28.052545 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:08:28.052545 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:08:28.052545 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:08:28.052545 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:08:28.052545 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:08:28.052545 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:08:28.052545 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:08:28.052545 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:08:28.052545 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 988f14d7-6354-4388-a51c-b4df9749c671 could not be found. Jul 19 11:08:28.052545 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:08:28.052545 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:08:28.052545 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:08:28.052545 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:08:28.052545 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:08:28.052545 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:08:28.052545 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:08:28.052545 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:08:28.052545 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:08:28.054023 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:08:28.054023 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 19 11:08:28.054023 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 19 11:08:28.054023 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:08:28.054023 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:08:28.054023 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:08:28.054023 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:08:28.054023 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 988f14d7-6354-4388-a51c-b4df9749c671 could not be found. Jul 19 11:08:28.054023 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:08:28.054023 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:08:28.054023 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:08:28.054023 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:08:28.054023 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:08:28.054023 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:08:28.054023 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:08:28.054023 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:08:28.054023 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 988f14d7-6354-4388-a51c-b4df9749c671 could not be found. Jul 19 11:08:28.054023 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:08:28.055245 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-14389566-f3aa-41d7-b574-99fd4fc39c47 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Returning 404 to user: Instance 988f14d7-6354-4388-a51c-b4df9749c671 could not be found. {{(pid=117849) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:08:28.055329 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-45947513-8b33-4821-b2a5-84fc23cad035 req-c4798317-92c1-49eb-a143-f8ecf94a34ba service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f52fbfae-577b-403d-a4d3-9a4b79ae2326", "name": "network-vif-plugged", "status": "completed", "tag": "748a7315-79e9-4ab8-ad6a-336c057e8b2c"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:08:28.058689 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-14389566-f3aa-41d7-b574-99fd4fc39c47 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671" status: 404 len: 111 microversion: 2.1 time: 0.064323 Jul 19 11:08:28.059408 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 313/646] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:27 2022] GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671 => generated 111 bytes in 69 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:08:28.068022 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-677d5fe8-eb4a-44cb-b73a-781a35f0385e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '.version_select at 0x7f04aa497d30>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:28.074993 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-45947513-8b33-4821-b2a5-84fc23cad035 req-c4798317-92c1-49eb-a143-f8ecf94a34ba service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:28.077109 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-45947513-8b33-4821-b2a5-84fc23cad035 req-c4798317-92c1-49eb-a143-f8ecf94a34ba service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:28.094107 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-45947513-8b33-4821-b2a5-84fc23cad035 req-c4798317-92c1-49eb-a143-f8ecf94a34ba service nova] Creating event network-vif-plugged:748a7315-79e9-4ab8-ad6a-336c057e8b2c for instance f52fbfae-577b-403d-a4d3-9a4b79ae2326 on hv-cinder-maste Jul 19 11:08:28.101524 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-45947513-8b33-4821-b2a5-84fc23cad035 req-c4798317-92c1-49eb-a143-f8ecf94a34ba service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.051739 Jul 19 11:08:28.102492 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 334/647] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:08:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:08:28.102633 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-677d5fe8-eb4a-44cb-b73a-781a35f0385e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-40836903" status: 202 len: 0 microversion: 2.1 time: 0.038229 Jul 19 11:08:28.105033 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 314/648] 10.222.0.33 () {58 vars in 1214 bytes} [Tue Jul 19 11:08:28 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-40836903 => generated 0 bytes in 41 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 19 11:08:28.145746 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-64747fd6-9f74-47f6-92c3-640e7ef3b46a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:28.150074 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-64747fd6-9f74-47f6-92c3-640e7ef3b46a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:28.166686 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-64747fd6-9f74-47f6-92c3-640e7ef3b46a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:28.167981 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-64747fd6-9f74-47f6-92c3-640e7ef3b46a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:28.169039 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-64747fd6-9f74-47f6-92c3-640e7ef3b46a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:28.169748 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-64747fd6-9f74-47f6-92c3-640e7ef3b46a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:28.458605 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-64747fd6-9f74-47f6-92c3-640e7ef3b46a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:28.467591 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-64747fd6-9f74-47f6-92c3-640e7ef3b46a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:28.469802 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-64747fd6-9f74-47f6-92c3-640e7ef3b46a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:28.475588 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-64747fd6-9f74-47f6-92c3-640e7ef3b46a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1807 microversion: 2.20 time: 0.340014 Jul 19 11:08:28.486003 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 335/649] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:08:28 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1807 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:08:28.514229 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-9211f20c-c791-4cac-91e1-0399f91e5db7 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:28.515012 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-9211f20c-c791-4cac-91e1-0399f91e5db7 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 12cd8c55-b30f-40ee-8cf5-7c974b8caafb] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:28.525757 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9211f20c-c791-4cac-91e1-0399f91e5db7 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:28.528314 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9211f20c-c791-4cac-91e1-0399f91e5db7 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:28.532201 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9211f20c-c791-4cac-91e1-0399f91e5db7 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:28.532808 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9211f20c-c791-4cac-91e1-0399f91e5db7 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:28.834252 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-6dbff13c-dcf6-4e22-a41b-2df80dc9affe req-13580cdf-c622-4341-906f-5f9afe053733 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "12cd8c55-b30f-40ee-8cf5-7c974b8caafb", "tag": "2648a82a-50bf-410f-86dd-82693a421316"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:08:28.860458 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-6dbff13c-dcf6-4e22-a41b-2df80dc9affe req-13580cdf-c622-4341-906f-5f9afe053733 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:28.860957 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-6dbff13c-dcf6-4e22-a41b-2df80dc9affe req-13580cdf-c622-4341-906f-5f9afe053733 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:28.895156 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-6dbff13c-dcf6-4e22-a41b-2df80dc9affe req-13580cdf-c622-4341-906f-5f9afe053733 service nova] Creating event network-changed:2648a82a-50bf-410f-86dd-82693a421316 for instance 12cd8c55-b30f-40ee-8cf5-7c974b8caafb on hv-cinder-maste Jul 19 11:08:28.907942 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-6dbff13c-dcf6-4e22-a41b-2df80dc9affe req-13580cdf-c622-4341-906f-5f9afe053733 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.084374 Jul 19 11:08:28.908572 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 336/650] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:08:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:08:29.026877 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-9211f20c-c791-4cac-91e1-0399f91e5db7 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:29.034602 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-9211f20c-c791-4cac-91e1-0399f91e5db7 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb" status: 200 len: 1278 microversion: 2.1 time: 0.524361 Jul 19 11:08:29.035179 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 315/651] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:28 2022] GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb => generated 1278 bytes in 526 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:29.391550 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-607bde13-bb77-40d1-969e-487a2d05fb96 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:29.392200 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-607bde13-bb77-40d1-969e-487a2d05fb96 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:29.399709 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-607bde13-bb77-40d1-969e-487a2d05fb96 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:29.400228 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-607bde13-bb77-40d1-969e-487a2d05fb96 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:29.401091 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-607bde13-bb77-40d1-969e-487a2d05fb96 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:29.401606 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-607bde13-bb77-40d1-969e-487a2d05fb96 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:29.493311 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-36a10aad-ec80-4edf-99e7-8a623a98b082 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:29.493921 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-36a10aad-ec80-4edf-99e7-8a623a98b082 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:29.500599 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-36a10aad-ec80-4edf-99e7-8a623a98b082 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:29.501294 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-36a10aad-ec80-4edf-99e7-8a623a98b082 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:29.502245 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-36a10aad-ec80-4edf-99e7-8a623a98b082 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:29.502839 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-36a10aad-ec80-4edf-99e7-8a623a98b082 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:29.529289 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.objects.instance [None req-607bde13-bb77-40d1-969e-487a2d05fb96 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lazy-loading 'services' on Instance uuid 4c71f564-de63-4fcd-88ce-01ecba3d8372 {{(pid=117848) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 19 11:08:29.637733 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-36a10aad-ec80-4edf-99e7-8a623a98b082 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:29.647859 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-36a10aad-ec80-4edf-99e7-8a623a98b082 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:29.649697 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-36a10aad-ec80-4edf-99e7-8a623a98b082 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:29.651940 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-36a10aad-ec80-4edf-99e7-8a623a98b082 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1807 microversion: 2.20 time: 0.162818 Jul 19 11:08:29.652260 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-607bde13-bb77-40d1-969e-487a2d05fb96 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "DELETE /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372/os-volume_attachments/45d056f5-25af-475a-8cf5-ff31a13487c2" status: 202 len: 0 microversion: 2.1 time: 0.264828 Jul 19 11:08:29.652924 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 316/652] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:08:29 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1807 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:08:29.653332 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 337/653] 10.222.0.33 () {58 vars in 1415 bytes} [Tue Jul 19 11:08:29 2022] DELETE /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372/os-volume_attachments/45d056f5-25af-475a-8cf5-ff31a13487c2 => generated 0 bytes in 267 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 19 11:08:29.669324 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-c7d194c8-8978-4df6-a01e-3b255c76e765 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:29.670852 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-c7d194c8-8978-4df6-a01e-3b255c76e765 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:29.677967 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c7d194c8-8978-4df6-a01e-3b255c76e765 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:29.678531 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c7d194c8-8978-4df6-a01e-3b255c76e765 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:29.679442 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c7d194c8-8978-4df6-a01e-3b255c76e765 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:29.679942 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c7d194c8-8978-4df6-a01e-3b255c76e765 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:29.712055 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-c7d194c8-8978-4df6-a01e-3b255c76e765 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047510 Jul 19 11:08:29.712772 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 317/654] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:08:29 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:08:30.052180 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-5a72cfa0-674b-482c-bb05-c210ae1f8a93 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:30.052839 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-5a72cfa0-674b-482c-bb05-c210ae1f8a93 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 12cd8c55-b30f-40ee-8cf5-7c974b8caafb] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:30.059899 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5a72cfa0-674b-482c-bb05-c210ae1f8a93 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:30.060348 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5a72cfa0-674b-482c-bb05-c210ae1f8a93 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:30.061137 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5a72cfa0-674b-482c-bb05-c210ae1f8a93 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:30.061572 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5a72cfa0-674b-482c-bb05-c210ae1f8a93 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:30.192812 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-5a72cfa0-674b-482c-bb05-c210ae1f8a93 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:30.196909 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=1,nova_cell1:INSERT=2 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:08:30.201663 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-5a72cfa0-674b-482c-bb05-c210ae1f8a93 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb" status: 200 len: 1278 microversion: 2.1 time: 0.153780 Jul 19 11:08:30.202232 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 338/655] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:30 2022] GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb => generated 1278 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:30.670721 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-7fc3f27c-1fb2-4c2c-acbe-17e0cc16b886 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:30.671279 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-7fc3f27c-1fb2-4c2c-acbe-17e0cc16b886 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:30.677661 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7fc3f27c-1fb2-4c2c-acbe-17e0cc16b886 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:30.678111 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7fc3f27c-1fb2-4c2c-acbe-17e0cc16b886 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:30.678867 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7fc3f27c-1fb2-4c2c-acbe-17e0cc16b886 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:30.679300 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7fc3f27c-1fb2-4c2c-acbe-17e0cc16b886 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:30.730310 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-048a17d9-6833-45ec-8f4a-a9ef1fd0c431 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:30.731298 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-048a17d9-6833-45ec-8f4a-a9ef1fd0c431 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:30.744653 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-048a17d9-6833-45ec-8f4a-a9ef1fd0c431 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:30.745183 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-048a17d9-6833-45ec-8f4a-a9ef1fd0c431 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:30.746145 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-048a17d9-6833-45ec-8f4a-a9ef1fd0c431 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:30.746579 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-048a17d9-6833-45ec-8f4a-a9ef1fd0c431 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:30.776208 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-048a17d9-6833-45ec-8f4a-a9ef1fd0c431 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053609 Jul 19 11:08:30.777000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 339/656] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:08:30 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:08:30.809792 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-7fc3f27c-1fb2-4c2c-acbe-17e0cc16b886 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:30.817998 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-7fc3f27c-1fb2-4c2c-acbe-17e0cc16b886 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:30.819644 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-7fc3f27c-1fb2-4c2c-acbe-17e0cc16b886 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:30.821332 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-7fc3f27c-1fb2-4c2c-acbe-17e0cc16b886 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1807 microversion: 2.20 time: 0.154760 Jul 19 11:08:30.822159 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 318/657] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:08:30 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1807 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:08:31.218211 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-9db1c5c5-604e-4aa4-954a-963bfb193d3d tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:31.218827 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-9db1c5c5-604e-4aa4-954a-963bfb193d3d tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 12cd8c55-b30f-40ee-8cf5-7c974b8caafb] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:31.224996 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9db1c5c5-604e-4aa4-954a-963bfb193d3d tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:31.225484 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9db1c5c5-604e-4aa4-954a-963bfb193d3d tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:31.226357 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9db1c5c5-604e-4aa4-954a-963bfb193d3d tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:31.226831 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9db1c5c5-604e-4aa4-954a-963bfb193d3d tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:31.366865 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-9db1c5c5-604e-4aa4-954a-963bfb193d3d tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:31.377614 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-9db1c5c5-604e-4aa4-954a-963bfb193d3d tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb" status: 200 len: 1278 microversion: 2.1 time: 0.163081 Jul 19 11:08:31.378195 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 340/658] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:31 2022] GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb => generated 1278 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:31.788080 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-7c096966-5e1e-46ed-b0bf-eb30e484b6b2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:31.788956 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-7c096966-5e1e-46ed-b0bf-eb30e484b6b2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:31.795405 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7c096966-5e1e-46ed-b0bf-eb30e484b6b2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:31.795851 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7c096966-5e1e-46ed-b0bf-eb30e484b6b2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:31.796608 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7c096966-5e1e-46ed-b0bf-eb30e484b6b2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:31.797073 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7c096966-5e1e-46ed-b0bf-eb30e484b6b2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:31.825188 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-7c096966-5e1e-46ed-b0bf-eb30e484b6b2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041672 Jul 19 11:08:31.825721 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 319/659] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:08:31 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:08:31.839123 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-9ed974f9-cd64-45cf-8acc-c62361bea42f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:31.839727 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-9ed974f9-cd64-45cf-8acc-c62361bea42f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:31.846086 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9ed974f9-cd64-45cf-8acc-c62361bea42f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:31.846523 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9ed974f9-cd64-45cf-8acc-c62361bea42f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:31.847270 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9ed974f9-cd64-45cf-8acc-c62361bea42f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:31.847750 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9ed974f9-cd64-45cf-8acc-c62361bea42f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:31.975318 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-9ed974f9-cd64-45cf-8acc-c62361bea42f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:31.983653 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-9ed974f9-cd64-45cf-8acc-c62361bea42f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:31.985690 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-9ed974f9-cd64-45cf-8acc-c62361bea42f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:31.986840 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-9ed974f9-cd64-45cf-8acc-c62361bea42f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1807 microversion: 2.20 time: 0.151579 Jul 19 11:08:31.987404 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 341/660] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:08:31 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1807 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:08:32.395020 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-cd0fb2bb-6ae7-46a0-87b7-e0aea9acbf84 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:32.395675 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-cd0fb2bb-6ae7-46a0-87b7-e0aea9acbf84 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 12cd8c55-b30f-40ee-8cf5-7c974b8caafb] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:32.402223 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cd0fb2bb-6ae7-46a0-87b7-e0aea9acbf84 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:32.402675 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cd0fb2bb-6ae7-46a0-87b7-e0aea9acbf84 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:32.403430 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cd0fb2bb-6ae7-46a0-87b7-e0aea9acbf84 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:32.403887 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cd0fb2bb-6ae7-46a0-87b7-e0aea9acbf84 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:32.534216 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-cd0fb2bb-6ae7-46a0-87b7-e0aea9acbf84 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:32.551030 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-cd0fb2bb-6ae7-46a0-87b7-e0aea9acbf84 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb" status: 200 len: 1278 microversion: 2.1 time: 0.159863 Jul 19 11:08:32.551624 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 320/661] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:32 2022] GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb => generated 1278 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:32.836419 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-e6dbdf31-b01d-4972-8248-080a2ca1cc7e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:32.837751 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-e6dbdf31-b01d-4972-8248-080a2ca1cc7e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:32.844260 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e6dbdf31-b01d-4972-8248-080a2ca1cc7e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:32.844847 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e6dbdf31-b01d-4972-8248-080a2ca1cc7e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:32.845482 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e6dbdf31-b01d-4972-8248-080a2ca1cc7e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:32.845910 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e6dbdf31-b01d-4972-8248-080a2ca1cc7e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:32.873976 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-e6dbdf31-b01d-4972-8248-080a2ca1cc7e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041918 Jul 19 11:08:32.874524 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 342/662] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:08:32 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:08:33.005493 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-c7d5ba98-6373-43e0-a5d7-23b162762406 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:33.006691 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-c7d5ba98-6373-43e0-a5d7-23b162762406 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:33.009328 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell0:SELECT=4 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:08:33.014638 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c7d5ba98-6373-43e0-a5d7-23b162762406 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:33.015085 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c7d5ba98-6373-43e0-a5d7-23b162762406 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:33.015844 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c7d5ba98-6373-43e0-a5d7-23b162762406 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:33.016280 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c7d5ba98-6373-43e0-a5d7-23b162762406 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:33.067625 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell0:SELECT=8 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:08:33.146904 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-c7d5ba98-6373-43e0-a5d7-23b162762406 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:33.156205 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-c7d5ba98-6373-43e0-a5d7-23b162762406 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:33.158576 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-c7d5ba98-6373-43e0-a5d7-23b162762406 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:33.160040 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-c7d5ba98-6373-43e0-a5d7-23b162762406 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1794 microversion: 2.20 time: 0.158295 Jul 19 11:08:33.160671 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 321/663] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:08:33 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1794 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:08:33.567332 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-879b1db5-6ff7-4e45-ad90-33ff4d69dbc8 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:33.568014 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-879b1db5-6ff7-4e45-ad90-33ff4d69dbc8 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 12cd8c55-b30f-40ee-8cf5-7c974b8caafb] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:33.574123 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-879b1db5-6ff7-4e45-ad90-33ff4d69dbc8 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:33.574651 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-879b1db5-6ff7-4e45-ad90-33ff4d69dbc8 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:33.575483 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-879b1db5-6ff7-4e45-ad90-33ff4d69dbc8 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:33.575985 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-879b1db5-6ff7-4e45-ad90-33ff4d69dbc8 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:33.750928 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-879b1db5-6ff7-4e45-ad90-33ff4d69dbc8 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:33.760383 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-879b1db5-6ff7-4e45-ad90-33ff4d69dbc8 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb" status: 200 len: 1278 microversion: 2.1 time: 0.196793 Jul 19 11:08:33.761175 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 343/664] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:33 2022] GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb => generated 1278 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:33.885623 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-88dbca54-9aa3-4805-bc15-3e70c4459c64 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:33.886454 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-88dbca54-9aa3-4805-bc15-3e70c4459c64 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:33.892430 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-88dbca54-9aa3-4805-bc15-3e70c4459c64 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:33.892925 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-88dbca54-9aa3-4805-bc15-3e70c4459c64 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:33.893674 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-88dbca54-9aa3-4805-bc15-3e70c4459c64 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:33.894103 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-88dbca54-9aa3-4805-bc15-3e70c4459c64 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:33.924457 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-88dbca54-9aa3-4805-bc15-3e70c4459c64 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042806 Jul 19 11:08:33.925032 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 322/665] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:08:33 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:08:34.175586 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-677a7c91-ea73-4d58-ab33-6e424c444737 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:34.176195 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-677a7c91-ea73-4d58-ab33-6e424c444737 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:34.185083 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-677a7c91-ea73-4d58-ab33-6e424c444737 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:34.185535 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-677a7c91-ea73-4d58-ab33-6e424c444737 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:34.186300 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-677a7c91-ea73-4d58-ab33-6e424c444737 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:34.186735 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-677a7c91-ea73-4d58-ab33-6e424c444737 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:34.320263 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-677a7c91-ea73-4d58-ab33-6e424c444737 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:34.328283 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-677a7c91-ea73-4d58-ab33-6e424c444737 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1594 microversion: 2.1 time: 0.156495 Jul 19 11:08:34.328900 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 344/666] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:34 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1594 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:34.776408 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-54431b1e-7055-4120-95b5-5be93ce21bff tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:34.777026 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-54431b1e-7055-4120-95b5-5be93ce21bff tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 12cd8c55-b30f-40ee-8cf5-7c974b8caafb] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:34.783553 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-54431b1e-7055-4120-95b5-5be93ce21bff tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:34.783998 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-54431b1e-7055-4120-95b5-5be93ce21bff tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:34.784791 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-54431b1e-7055-4120-95b5-5be93ce21bff tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:34.785224 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-54431b1e-7055-4120-95b5-5be93ce21bff tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:34.922990 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-54431b1e-7055-4120-95b5-5be93ce21bff tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:34.938380 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-35a1fea5-1030-4935-99f6-90bab2e323d5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:34.938530 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-54431b1e-7055-4120-95b5-5be93ce21bff tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb" status: 200 len: 1266 microversion: 2.1 time: 0.165817 Jul 19 11:08:34.939716 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-35a1fea5-1030-4935-99f6-90bab2e323d5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:34.947776 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-35a1fea5-1030-4935-99f6-90bab2e323d5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:34.948413 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-35a1fea5-1030-4935-99f6-90bab2e323d5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:34.948854 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 323/667] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:34 2022] GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb => generated 1266 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:34.949209 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-35a1fea5-1030-4935-99f6-90bab2e323d5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:34.949809 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-35a1fea5-1030-4935-99f6-90bab2e323d5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:34.979062 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-35a1fea5-1030-4935-99f6-90bab2e323d5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047035 Jul 19 11:08:34.979634 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 345/668] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:08:34 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:08:35.956260 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-bbcf3322-ddb8-4d16-9243-8509d7595391 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:35.956900 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-bbcf3322-ddb8-4d16-9243-8509d7595391 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 12cd8c55-b30f-40ee-8cf5-7c974b8caafb] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:35.964164 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bbcf3322-ddb8-4d16-9243-8509d7595391 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:35.964614 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bbcf3322-ddb8-4d16-9243-8509d7595391 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:35.965421 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bbcf3322-ddb8-4d16-9243-8509d7595391 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:35.965857 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bbcf3322-ddb8-4d16-9243-8509d7595391 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:35.990662 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-7c68859a-609c-434b-bf76-975bc7d90285 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:35.991586 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-7c68859a-609c-434b-bf76-975bc7d90285 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:35.998652 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7c68859a-609c-434b-bf76-975bc7d90285 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:35.999101 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7c68859a-609c-434b-bf76-975bc7d90285 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:35.999880 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7c68859a-609c-434b-bf76-975bc7d90285 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:36.000311 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7c68859a-609c-434b-bf76-975bc7d90285 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:36.031335 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-7c68859a-609c-434b-bf76-975bc7d90285 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.044519 Jul 19 11:08:36.031931 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 346/669] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:08:35 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372/os-volume_attachments => generated 25 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Jul 19 11:08:36.095918 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-44f6c654-7a46-4d91-8ad4-ee6999e05e3f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:36.096580 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-44f6c654-7a46-4d91-8ad4-ee6999e05e3f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:36.104256 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-44f6c654-7a46-4d91-8ad4-ee6999e05e3f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:36.104905 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-44f6c654-7a46-4d91-8ad4-ee6999e05e3f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:36.105815 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-44f6c654-7a46-4d91-8ad4-ee6999e05e3f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:36.106354 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-44f6c654-7a46-4d91-8ad4-ee6999e05e3f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:36.134275 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-44f6c654-7a46-4d91-8ad4-ee6999e05e3f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Going to try to terminate instance {{(pid=117848) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 19 11:08:36.212591 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_api:SELECT=98,nova_api:UPDATE=2 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:08:36.224224 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-44f6c654-7a46-4d91-8ad4-ee6999e05e3f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "DELETE /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 204 len: 0 microversion: 2.1 time: 0.132848 Jul 19 11:08:36.229344 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 347/670] 10.222.0.33 () {58 vars in 1238 bytes} [Tue Jul 19 11:08:36 2022] DELETE /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 19 11:08:36.236683 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-cda83a7d-a828-41da-b135-4f6b0a299620 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:36.237732 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-cda83a7d-a828-41da-b135-4f6b0a299620 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:36.245164 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cda83a7d-a828-41da-b135-4f6b0a299620 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:36.245610 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cda83a7d-a828-41da-b135-4f6b0a299620 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:36.246569 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cda83a7d-a828-41da-b135-4f6b0a299620 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:36.247005 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cda83a7d-a828-41da-b135-4f6b0a299620 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:36.399865 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-bbcf3322-ddb8-4d16-9243-8509d7595391 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:36.408354 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-bbcf3322-ddb8-4d16-9243-8509d7595391 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb" status: 200 len: 1266 microversion: 2.1 time: 0.455880 Jul 19 11:08:36.418882 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-cda83a7d-a828-41da-b135-4f6b0a299620 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:36.420100 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 324/671] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:35 2022] GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb => generated 1266 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:36.425007 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-7da65c6f-2914-447f-8ca4-ff88c44ccfca req-122890f2-f8c4-497f-8d68-237574fed6a9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "03624642-ce02-450f-821d-1c135e5d7a4a", "tag": "6b6d7587-f2e5-4c60-8cf8-d1103dc5c0b6"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:08:36.429376 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-cda83a7d-a828-41da-b135-4f6b0a299620 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1682 microversion: 2.1 time: 0.197959 Jul 19 11:08:36.430858 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 348/672] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:36 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1682 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:36.454533 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-7da65c6f-2914-447f-8ca4-ff88c44ccfca req-122890f2-f8c4-497f-8d68-237574fed6a9 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:36.455113 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-7da65c6f-2914-447f-8ca4-ff88c44ccfca req-122890f2-f8c4-497f-8d68-237574fed6a9 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:36.459390 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=100 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:08:36.469261 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-7da65c6f-2914-447f-8ca4-ff88c44ccfca req-122890f2-f8c4-497f-8d68-237574fed6a9 service nova] Creating event network-changed:6b6d7587-f2e5-4c60-8cf8-d1103dc5c0b6 for instance 03624642-ce02-450f-821d-1c135e5d7a4a on hv-cinder-maste Jul 19 11:08:36.475274 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-7da65c6f-2914-447f-8ca4-ff88c44ccfca req-122890f2-f8c4-497f-8d68-237574fed6a9 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.053300 Jul 19 11:08:36.475992 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 325/673] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:08:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:08:37.322567 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-b07ceb6f-41fa-44fa-aa77-b9db582c0d43 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-1499670947"}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:08:37.323109 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-b07ceb6f-41fa-44fa-aa77-b9db582c0d43 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:37.329703 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b07ceb6f-41fa-44fa-aa77-b9db582c0d43 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:37.330150 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b07ceb6f-41fa-44fa-aa77-b9db582c0d43 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:37.330919 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b07ceb6f-41fa-44fa-aa77-b9db582c0d43 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:37.331532 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b07ceb6f-41fa-44fa-aa77-b9db582c0d43 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:37.427380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-8c042981-f6c8-405a-98d0-ef66608b4401 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:37.428516 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-8c042981-f6c8-405a-98d0-ef66608b4401 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 12cd8c55-b30f-40ee-8cf5-7c974b8caafb] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:37.435239 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8c042981-f6c8-405a-98d0-ef66608b4401 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:37.435724 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8c042981-f6c8-405a-98d0-ef66608b4401 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:37.436510 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8c042981-f6c8-405a-98d0-ef66608b4401 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:37.436983 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8c042981-f6c8-405a-98d0-ef66608b4401 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:37.464018 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.network.security_group_api [None req-b07ceb6f-41fa-44fa-aa77-b9db582c0d43 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Adding security group 0c90f2f3-5e80-4c64-b49b-9025a5d2bb48 to port 6b6d7587-f2e5-4c60-8cf8-d1103dc5c0b6 Jul 19 11:08:37.571567 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-8c042981-f6c8-405a-98d0-ef66608b4401 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:37.579227 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-8c042981-f6c8-405a-98d0-ef66608b4401 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb" status: 200 len: 1266 microversion: 2.1 time: 0.156117 Jul 19 11:08:37.579787 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 326/674] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:37 2022] GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb => generated 1266 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:37.584720 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-025bc51b-f3c8-4918-a71b-e59a78d13713 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:37.585316 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-025bc51b-f3c8-4918-a71b-e59a78d13713 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:37.591118 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-025bc51b-f3c8-4918-a71b-e59a78d13713 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:37.591558 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-025bc51b-f3c8-4918-a71b-e59a78d13713 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:37.592310 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-025bc51b-f3c8-4918-a71b-e59a78d13713 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:37.592762 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-025bc51b-f3c8-4918-a71b-e59a78d13713 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:37.735783 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-025bc51b-f3c8-4918-a71b-e59a78d13713 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:37.743625 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-025bc51b-f3c8-4918-a71b-e59a78d13713 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1682 microversion: 2.1 time: 0.162889 Jul 19 11:08:37.744194 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 327/675] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:37 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1682 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:37.841901 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-b07ceb6f-41fa-44fa-aa77-b9db582c0d43 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "POST /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a/action" status: 202 len: 0 microversion: 2.1 time: 0.523058 Jul 19 11:08:37.843092 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 349/676] 10.222.0.33 () {60 vars in 1277 bytes} [Tue Jul 19 11:08:37 2022] POST /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a/action => generated 0 bytes in 525 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 19 11:08:37.851801 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-0584ae31-116d-4af8-ba55-eb0b9bbcb1c4 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:37.853267 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-0584ae31-116d-4af8-ba55-eb0b9bbcb1c4 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:37.863864 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0584ae31-116d-4af8-ba55-eb0b9bbcb1c4 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:37.865176 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0584ae31-116d-4af8-ba55-eb0b9bbcb1c4 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:37.866996 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0584ae31-116d-4af8-ba55-eb0b9bbcb1c4 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:37.867685 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0584ae31-116d-4af8-ba55-eb0b9bbcb1c4 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:38.096210 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-0584ae31-116d-4af8-ba55-eb0b9bbcb1c4 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:38.104594 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-0584ae31-116d-4af8-ba55-eb0b9bbcb1c4 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1641 microversion: 2.1 time: 0.257329 Jul 19 11:08:38.105445 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 328/677] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:37 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1641 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:38.490891 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-7a39606e-1f1d-4538-9eee-ce2eb456c5fa req-35937142-f87f-4176-89c9-3cc78ce647f0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "03624642-ce02-450f-821d-1c135e5d7a4a", "tag": "6b6d7587-f2e5-4c60-8cf8-d1103dc5c0b6"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:08:38.505137 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-7a39606e-1f1d-4538-9eee-ce2eb456c5fa req-35937142-f87f-4176-89c9-3cc78ce647f0 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:38.507690 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-7a39606e-1f1d-4538-9eee-ce2eb456c5fa req-35937142-f87f-4176-89c9-3cc78ce647f0 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:38.522146 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-7a39606e-1f1d-4538-9eee-ce2eb456c5fa req-35937142-f87f-4176-89c9-3cc78ce647f0 service nova] Creating event network-changed:6b6d7587-f2e5-4c60-8cf8-d1103dc5c0b6 for instance 03624642-ce02-450f-821d-1c135e5d7a4a on hv-cinder-maste Jul 19 11:08:38.527085 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-7a39606e-1f1d-4538-9eee-ce2eb456c5fa req-35937142-f87f-4176-89c9-3cc78ce647f0 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041516 Jul 19 11:08:38.527879 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 350/678] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:08:38 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:08:38.597941 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-0d08312a-2508-4824-8b80-e4fc2aee5501 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:38.600506 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-0d08312a-2508-4824-8b80-e4fc2aee5501 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 12cd8c55-b30f-40ee-8cf5-7c974b8caafb] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:38.609962 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0d08312a-2508-4824-8b80-e4fc2aee5501 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:38.610555 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0d08312a-2508-4824-8b80-e4fc2aee5501 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:38.611559 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0d08312a-2508-4824-8b80-e4fc2aee5501 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:38.612211 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0d08312a-2508-4824-8b80-e4fc2aee5501 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:38.761730 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-6fe00e90-54c9-4e4f-a882-069fa82ccb72 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:38.762429 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-6fe00e90-54c9-4e4f-a882-069fa82ccb72 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:38.771632 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6fe00e90-54c9-4e4f-a882-069fa82ccb72 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:38.772115 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6fe00e90-54c9-4e4f-a882-069fa82ccb72 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:38.772922 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6fe00e90-54c9-4e4f-a882-069fa82ccb72 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:38.773993 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6fe00e90-54c9-4e4f-a882-069fa82ccb72 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:38.832605 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-0d08312a-2508-4824-8b80-e4fc2aee5501 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:38.841594 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-0d08312a-2508-4824-8b80-e4fc2aee5501 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb" status: 200 len: 1266 microversion: 2.1 time: 0.248768 Jul 19 11:08:38.842338 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 329/679] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:38 2022] GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb => generated 1266 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:39.055203 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-6fe00e90-54c9-4e4f-a882-069fa82ccb72 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:39.065690 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-6fe00e90-54c9-4e4f-a882-069fa82ccb72 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1682 microversion: 2.1 time: 0.308140 Jul 19 11:08:39.066288 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 351/680] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:38 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1682 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:39.858414 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-5501d1cf-2e4d-4f77-907b-3b4faa5aa62f tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:39.858982 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-5501d1cf-2e4d-4f77-907b-3b4faa5aa62f tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 12cd8c55-b30f-40ee-8cf5-7c974b8caafb] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:39.865906 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5501d1cf-2e4d-4f77-907b-3b4faa5aa62f tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:39.866236 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5501d1cf-2e4d-4f77-907b-3b4faa5aa62f tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:39.867148 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5501d1cf-2e4d-4f77-907b-3b4faa5aa62f tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:39.867854 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5501d1cf-2e4d-4f77-907b-3b4faa5aa62f tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:40.002595 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-5501d1cf-2e4d-4f77-907b-3b4faa5aa62f tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:40.011544 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-5501d1cf-2e4d-4f77-907b-3b4faa5aa62f tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb" status: 200 len: 1266 microversion: 2.1 time: 0.156800 Jul 19 11:08:40.012354 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 330/681] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:39 2022] GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb => generated 1266 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:40.083027 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-cf4ea010-0380-461a-a919-5093c5f0d8db tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:40.083643 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-cf4ea010-0380-461a-a919-5093c5f0d8db tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:40.090082 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cf4ea010-0380-461a-a919-5093c5f0d8db tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:40.090539 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cf4ea010-0380-461a-a919-5093c5f0d8db tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:40.091315 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cf4ea010-0380-461a-a919-5093c5f0d8db tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:40.092051 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cf4ea010-0380-461a-a919-5093c5f0d8db tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:40.127247 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cf4ea010-0380-461a-a919-5093c5f0d8db tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:40.127960 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cf4ea010-0380-461a-a919-5093c5f0d8db tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:40.128417 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.objects.instance [None req-cf4ea010-0380-461a-a919-5093c5f0d8db tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lazy-loading 'fault' on Instance uuid 4c71f564-de63-4fcd-88ce-01ecba3d8372 {{(pid=117848) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 19 11:08:40.170909 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-cf4ea010-0380-461a-a919-5093c5f0d8db tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:40.179150 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-cf4ea010-0380-461a-a919-5093c5f0d8db tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 200 len: 1331 microversion: 2.1 time: 0.100043 Jul 19 11:08:40.179769 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 352/682] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:40 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 1331 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:40.543888 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-c45c2deb-9670-4cc0-bf5f-d6481f1edbb3 req-5c1b4be2-b7a0-499e-8024-95d8e5857bfb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4c71f564-de63-4fcd-88ce-01ecba3d8372", "name": "network-vif-deleted", "tag": "526a30d7-5fcf-4c45-86bf-6ddbc902d48c"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:08:40.553288 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-c45c2deb-9670-4cc0-bf5f-d6481f1edbb3 req-5c1b4be2-b7a0-499e-8024-95d8e5857bfb service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:40.554060 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-c45c2deb-9670-4cc0-bf5f-d6481f1edbb3 req-5c1b4be2-b7a0-499e-8024-95d8e5857bfb service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:40.567380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-c45c2deb-9670-4cc0-bf5f-d6481f1edbb3 req-5c1b4be2-b7a0-499e-8024-95d8e5857bfb service nova] Creating event network-vif-deleted:526a30d7-5fcf-4c45-86bf-6ddbc902d48c for instance 4c71f564-de63-4fcd-88ce-01ecba3d8372 on hv-cinder-maste Jul 19 11:08:40.572640 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-c45c2deb-9670-4cc0-bf5f-d6481f1edbb3 req-5c1b4be2-b7a0-499e-8024-95d8e5857bfb service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032387 Jul 19 11:08:40.573293 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 331/683] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:08:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:08:41.028583 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-bf087d46-0198-45ff-a09b-0796bbe7f5bb tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:41.029207 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-bf087d46-0198-45ff-a09b-0796bbe7f5bb tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 12cd8c55-b30f-40ee-8cf5-7c974b8caafb] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:41.035401 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-bf087d46-0198-45ff-a09b-0796bbe7f5bb tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:41.035841 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-bf087d46-0198-45ff-a09b-0796bbe7f5bb tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:41.036587 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-bf087d46-0198-45ff-a09b-0796bbe7f5bb tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:41.037034 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-bf087d46-0198-45ff-a09b-0796bbe7f5bb tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:41.166574 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-bf087d46-0198-45ff-a09b-0796bbe7f5bb tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:41.174255 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-bf087d46-0198-45ff-a09b-0796bbe7f5bb tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb" status: 200 len: 1266 microversion: 2.1 time: 0.149458 Jul 19 11:08:41.174822 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 353/684] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:41 2022] GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb => generated 1266 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:41.196875 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-4d1de4e9-3190-43d9-a3b8-2f02bd4083ba tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:41.197622 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-4d1de4e9-3190-43d9-a3b8-2f02bd4083ba tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:41.203775 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4d1de4e9-3190-43d9-a3b8-2f02bd4083ba tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:41.204222 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4d1de4e9-3190-43d9-a3b8-2f02bd4083ba tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:41.205012 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4d1de4e9-3190-43d9-a3b8-2f02bd4083ba tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:41.205452 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4d1de4e9-3190-43d9-a3b8-2f02bd4083ba tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:41.218330 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.wsgi [None req-4d1de4e9-3190-43d9-a3b8-2f02bd4083ba tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] HTTP exception thrown: Instance 4c71f564-de63-4fcd-88ce-01ecba3d8372 could not be found. Jul 19 11:08:41.219900 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-4d1de4e9-3190-43d9-a3b8-2f02bd4083ba tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Request method failure captured: Jul 19 11:08:41.219900 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: request: GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 HTTP/1.1 Jul 19 11:08:41.219900 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept: application/json Jul 19 11:08:41.219900 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept-Encoding: identity Jul 19 11:08:41.219900 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Connection: close Jul 19 11:08:41.219900 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Length: 0 Jul 19 11:08:41.219900 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Type: application/json Jul 19 11:08:41.219900 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Host: 10.222.0.33 Jul 19 11:08:41.219900 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Openstack-System-Scope: None Jul 19 11:08:41.219900 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: User-Agent: python-urllib3/1.26.9 Jul 19 11:08:41.219900 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Auth-Token: gAAAAABi1o_HYpL7r_umyWTTKsnaQkfxgRQPrCVzXYV3YxPvpHKAUsKkXxPNkPyHBhEDbahQjtwakXlDpqARj9YKw3tFYmG0gCU4SGJZDCiqyWHloRDABrhK0mtm6JR4S49vBX-Zeg1oz_jYGWaYn6kU0GfIXikwtQO5JQ3VMjNcjqJ15uYlE9Q Jul 19 11:08:41.219900 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Id: None Jul 19 11:08:41.219900 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Name: None Jul 19 11:08:41.219900 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Identity-Status: Confirmed Jul 19 11:08:41.219900 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Is-Admin-Project: True Jul 19 11:08:41.219900 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Id: default Jul 19 11:08:41.219900 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Name: Default Jul 19 11:08:41.219900 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Id: ef534f1d06ea415d98857ec7f9e5fb97 Jul 19 11:08:41.219900 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Name: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:08:41.219900 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Role: member,reader Jul 19 11:08:41.219900 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Roles: member,reader Jul 19 11:08:41.221397 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:08:41.221397 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:08:41.221397 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Id: ef534f1d06ea415d98857ec7f9e5fb97 Jul 19 11:08:41.221397 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:08:41.221397 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User: tempest-AttachVolumeNegativeTest-434278602-project Jul 19 11:08:41.221397 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Id: default Jul 19 11:08:41.221397 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Name: Default Jul 19 11:08:41.221397 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Id: b8f2a78f3a3e4b8e8937e3bc39524690 Jul 19 11:08:41.221397 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Name: tempest-AttachVolumeNegativeTest-434278602-project Jul 19 11:08:41.221397 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: method: > Jul 19 11:08:41.221397 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: exception: Instance 4c71f564-de63-4fcd-88ce-01ecba3d8372 could not be found. Jul 19 11:08:41.221397 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: action_args: {'id': '4c71f564-de63-4fcd-88ce-01ecba3d8372'} Jul 19 11:08:41.222319 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:08:41.222319 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:08:41.222319 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:08:41.222319 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:08:41.222319 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:08:41.222319 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:08:41.222319 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:08:41.222319 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:08:41.222319 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:08:41.222319 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:08:41.222319 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4c71f564-de63-4fcd-88ce-01ecba3d8372 could not be found. Jul 19 11:08:41.222319 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:08:41.222319 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:08:41.222319 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:08:41.222319 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:08:41.222319 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:08:41.222319 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:08:41.222319 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:08:41.222319 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:08:41.222319 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:08:41.223732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:08:41.223732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 19 11:08:41.223732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 19 11:08:41.223732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:08:41.223732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:08:41.223732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:08:41.223732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:08:41.223732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4c71f564-de63-4fcd-88ce-01ecba3d8372 could not be found. Jul 19 11:08:41.223732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:08:41.223732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:08:41.223732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:08:41.223732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:08:41.223732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:08:41.223732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:08:41.223732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:08:41.223732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:08:41.223732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4c71f564-de63-4fcd-88ce-01ecba3d8372 could not be found. Jul 19 11:08:41.223732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:08:41.225101 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-4d1de4e9-3190-43d9-a3b8-2f02bd4083ba tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Returning 404 to user: Instance 4c71f564-de63-4fcd-88ce-01ecba3d8372 could not be found. {{(pid=117849) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:08:41.225101 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-4d1de4e9-3190-43d9-a3b8-2f02bd4083ba tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 404 len: 111 microversion: 2.1 time: 0.029767 Jul 19 11:08:41.225101 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 332/685] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:41 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:08:41.230064 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-c672ff77-08e3-4f02-bcbc-76c9ed0fe03a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '.version_select at 0x7f04aa460820>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:41.248707 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-c672ff77-08e3-4f02-bcbc-76c9ed0fe03a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1514182937" status: 202 len: 0 microversion: 2.1 time: 0.022203 Jul 19 11:08:41.249274 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 354/686] 10.222.0.33 () {58 vars in 1220 bytes} [Tue Jul 19 11:08:41 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1514182937 => generated 0 bytes in 24 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 19 11:08:41.885061 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-6ca934ad-dbf7-47f7-9782-214e40e074cd tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Action: 'create', calling method: .version_select at 0x7f04aa2981f0>, body: {"keypair": {"name": "tempest-keypair-1830923980", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKijBdByfdtLgClvIRgla1KNYAMoobJDEzHPPcHzrbRQdcIZbe6uuuWJal3BfSvjOcJ0wQl2BnPoYxfbdXPJZz8twK3P7WViKWQ+6NKBsGJeYyO5CV6VaK+oPU5z3dkGjA=="}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:08:41.889095 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:DELETE=1 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:08:41.902543 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-6ca934ad-dbf7-47f7-9782-214e40e074cd tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Getting quotas for project ef534f1d06ea415d98857ec7f9e5fb97. Resources: {'key_pairs'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:08:41.906258 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-6ca934ad-dbf7-47f7-9782-214e40e074cd tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Getting quotas for user b8f2a78f3a3e4b8e8937e3bc39524690 and project ef534f1d06ea415d98857ec7f9e5fb97. Resources: {'key_pairs'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:08:41.928247 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-6ca934ad-dbf7-47f7-9782-214e40e074cd tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.046975 Jul 19 11:08:41.929032 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 333/687] 10.222.0.33 () {60 vars in 1158 bytes} [Tue Jul 19 11:08:41 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:08:42.192891 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-bf702024-42e7-43ee-9698-354e62371003 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:42.195000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-bf702024-42e7-43ee-9698-354e62371003 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 12cd8c55-b30f-40ee-8cf5-7c974b8caafb] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:42.202357 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-bf702024-42e7-43ee-9698-354e62371003 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:42.203206 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-bf702024-42e7-43ee-9698-354e62371003 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:42.204237 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-bf702024-42e7-43ee-9698-354e62371003 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:42.205131 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-bf702024-42e7-43ee-9698-354e62371003 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:42.228162 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:08:42.374661 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-bf702024-42e7-43ee-9698-354e62371003 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:42.386160 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-bf702024-42e7-43ee-9698-354e62371003 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb" status: 200 len: 1266 microversion: 2.1 time: 0.198374 Jul 19 11:08:42.389443 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 355/688] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:42 2022] GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb => generated 1266 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:43.139735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-3da48780-7bec-4de6-bbb0-616c0d2c95f8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1244306607", "imageRef": "3085883e-2289-4314-aa47-9c41d4368b02", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "0b1fabad-b81a-4ee7-9fd9-62393ba0d37f"}], "security_groups": [{"name": "tempest-securitygroup--1390553303"}], "key_name": "tempest-keypair-1830923980", "user_data": "}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:08:43.239784 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.network.neutron [None req-3da48780-7bec-4de6-bbb0-616c0d2c95f8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] validate_networks() for [('0b1fabad-b81a-4ee7-9fd9-62393ba0d37f', None, None, None, None, None)] {{(pid=117849) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 19 11:08:43.405727 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-19c0a768-a1fd-426e-bf5c-613155585811 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:43.406241 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-19c0a768-a1fd-426e-bf5c-613155585811 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 12cd8c55-b30f-40ee-8cf5-7c974b8caafb] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:43.413461 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-19c0a768-a1fd-426e-bf5c-613155585811 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:43.414670 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-19c0a768-a1fd-426e-bf5c-613155585811 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:43.415022 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-19c0a768-a1fd-426e-bf5c-613155585811 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:43.415459 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-19c0a768-a1fd-426e-bf5c-613155585811 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:43.545157 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-19c0a768-a1fd-426e-bf5c-613155585811 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:43.554303 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-19c0a768-a1fd-426e-bf5c-613155585811 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb" status: 200 len: 1266 microversion: 2.1 time: 0.152449 Jul 19 11:08:43.557797 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 356/689] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:43 2022] GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb => generated 1266 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:43.627277 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-3da48780-7bec-4de6-bbb0-616c0d2c95f8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Flavor limits 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 19 11:08:43.627696 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-3da48780-7bec-4de6-bbb0-616c0d2c95f8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Image limits 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 19 11:08:43.628165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-3da48780-7bec-4de6-bbb0-616c0d2c95f8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Flavor pref 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 19 11:08:43.628569 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-3da48780-7bec-4de6-bbb0-616c0d2c95f8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Image pref 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 19 11:08:43.629007 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-3da48780-7bec-4de6-bbb0-616c0d2c95f8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 19 11:08:43.637781 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3da48780-7bec-4de6-bbb0-616c0d2c95f8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:43.638222 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3da48780-7bec-4de6-bbb0-616c0d2c95f8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:43.639005 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3da48780-7bec-4de6-bbb0-616c0d2c95f8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:43.639432 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3da48780-7bec-4de6-bbb0-616c0d2c95f8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:43.662242 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-3da48780-7bec-4de6-bbb0-616c0d2c95f8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Getting quotas for project ef534f1d06ea415d98857ec7f9e5fb97. Resources: {'cores', 'instances', 'ram'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:08:43.667690 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-3da48780-7bec-4de6-bbb0-616c0d2c95f8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Getting quotas for user b8f2a78f3a3e4b8e8937e3bc39524690 and project ef534f1d06ea415d98857ec7f9e5fb97. Resources: {'cores', 'instances', 'ram'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:08:43.679525 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-3da48780-7bec-4de6-bbb0-616c0d2c95f8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Going to run 1 instances... {{(pid=117849) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 19 11:08:43.681967 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-3da48780-7bec-4de6-bbb0-616c0d2c95f8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 53545a7c-c683-49d6-941d-4445f2b09379] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='3085883e-2289-4314-aa47-9c41d4368b02',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117849) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 19 11:08:43.708934 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-61c3039e-4234-45ae-84a2-0b8a8c459bd1 req-8d2b4bb0-0f68-414e-8a26-87ef555352d6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "12cd8c55-b30f-40ee-8cf5-7c974b8caafb", "name": "network-vif-plugged", "status": "completed", "tag": "2648a82a-50bf-410f-86dd-82693a421316"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:08:43.719654 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-61c3039e-4234-45ae-84a2-0b8a8c459bd1 req-8d2b4bb0-0f68-414e-8a26-87ef555352d6 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:43.720824 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-61c3039e-4234-45ae-84a2-0b8a8c459bd1 req-8d2b4bb0-0f68-414e-8a26-87ef555352d6 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:43.732949 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-61c3039e-4234-45ae-84a2-0b8a8c459bd1 req-8d2b4bb0-0f68-414e-8a26-87ef555352d6 service nova] Creating event network-vif-plugged:2648a82a-50bf-410f-86dd-82693a421316 for instance 12cd8c55-b30f-40ee-8cf5-7c974b8caafb on hv-cinder-maste Jul 19 11:08:43.741340 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-61c3039e-4234-45ae-84a2-0b8a8c459bd1 req-8d2b4bb0-0f68-414e-8a26-87ef555352d6 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037788 Jul 19 11:08:43.742517 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 357/690] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:08:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:08:43.744161 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-3da48780-7bec-4de6-bbb0-616c0d2c95f8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.609061 Jul 19 11:08:43.749420 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 334/691] 10.222.0.33 () {60 vars in 1146 bytes} [Tue Jul 19 11:08:43 2022] POST /compute/v2.1/servers => generated 402 bytes in 611 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 19 11:08:43.755535 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-c35f3eb7-aa5d-47bb-a067-d0cddd17ced8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:43.756800 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-c35f3eb7-aa5d-47bb-a067-d0cddd17ced8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 53545a7c-c683-49d6-941d-4445f2b09379] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:43.813848 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-c35f3eb7-aa5d-47bb-a067-d0cddd17ced8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:43.822345 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-c35f3eb7-aa5d-47bb-a067-d0cddd17ced8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379" status: 200 len: 1242 microversion: 2.1 time: 0.070770 Jul 19 11:08:43.823624 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 358/692] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:43 2022] GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379 => generated 1242 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:43.860384 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-a35eaaa7-bfe2-45a1-8a03-fda6b0e9bc84 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "5ac37faf-6388-4d0e-9939-097bb5b52afc"}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:08:43.861110 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-a35eaaa7-bfe2-45a1-8a03-fda6b0e9bc84 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:43.868271 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a35eaaa7-bfe2-45a1-8a03-fda6b0e9bc84 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:43.868755 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a35eaaa7-bfe2-45a1-8a03-fda6b0e9bc84 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:43.869538 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a35eaaa7-bfe2-45a1-8a03-fda6b0e9bc84 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:43.870000 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a35eaaa7-bfe2-45a1-8a03-fda6b0e9bc84 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:44.171447 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-a35eaaa7-bfe2-45a1-8a03-fda6b0e9bc84 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "POST /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.316214 Jul 19 11:08:44.172149 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 335/693] 10.222.0.33 () {60 vars in 1322 bytes} [Tue Jul 19 11:08:43 2022] POST /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3/os-volume_attachments => generated 194 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:08:44.574569 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-fe13b3e1-13aa-42c1-8178-0d3723befb30 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:44.575175 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-fe13b3e1-13aa-42c1-8178-0d3723befb30 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 12cd8c55-b30f-40ee-8cf5-7c974b8caafb] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:44.581923 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fe13b3e1-13aa-42c1-8178-0d3723befb30 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:44.582369 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fe13b3e1-13aa-42c1-8178-0d3723befb30 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:44.583120 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fe13b3e1-13aa-42c1-8178-0d3723befb30 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:44.583559 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fe13b3e1-13aa-42c1-8178-0d3723befb30 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:44.711003 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-fe13b3e1-13aa-42c1-8178-0d3723befb30 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:44.718369 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-fe13b3e1-13aa-42c1-8178-0d3723befb30 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb" status: 200 len: 1266 microversion: 2.1 time: 0.147703 Jul 19 11:08:44.718937 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 359/694] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:44 2022] GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb => generated 1266 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:44.839853 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-bdb5a61b-f626-46e1-b00a-8dbb0240b734 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:44.840543 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-bdb5a61b-f626-46e1-b00a-8dbb0240b734 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 53545a7c-c683-49d6-941d-4445f2b09379] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:44.847253 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bdb5a61b-f626-46e1-b00a-8dbb0240b734 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:44.847701 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bdb5a61b-f626-46e1-b00a-8dbb0240b734 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:44.848458 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bdb5a61b-f626-46e1-b00a-8dbb0240b734 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:44.848935 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bdb5a61b-f626-46e1-b00a-8dbb0240b734 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:44.911501 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-bdb5a61b-f626-46e1-b00a-8dbb0240b734 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:44.919821 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-bdb5a61b-f626-46e1-b00a-8dbb0240b734 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379" status: 200 len: 1238 microversion: 2.1 time: 0.084079 Jul 19 11:08:44.920739 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 336/695] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:44 2022] GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379 => generated 1238 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:45.735067 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-29cfd7fd-2c86-441e-91ab-57f424fc840a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:45.735704 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-29cfd7fd-2c86-441e-91ab-57f424fc840a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 12cd8c55-b30f-40ee-8cf5-7c974b8caafb] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:45.743798 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-29cfd7fd-2c86-441e-91ab-57f424fc840a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:45.744250 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-29cfd7fd-2c86-441e-91ab-57f424fc840a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:45.745057 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-29cfd7fd-2c86-441e-91ab-57f424fc840a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:45.745903 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-29cfd7fd-2c86-441e-91ab-57f424fc840a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:45.886067 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-29cfd7fd-2c86-441e-91ab-57f424fc840a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:45.893816 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-29cfd7fd-2c86-441e-91ab-57f424fc840a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb" status: 200 len: 1453 microversion: 2.1 time: 0.162672 Jul 19 11:08:45.894440 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 360/696] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:45 2022] GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb => generated 1453 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:45.909532 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-354bccb4-8989-44f0-bbda-421636b3e08a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:45.910134 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-354bccb4-8989-44f0-bbda-421636b3e08a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 12cd8c55-b30f-40ee-8cf5-7c974b8caafb] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:45.916523 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-354bccb4-8989-44f0-bbda-421636b3e08a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:45.917004 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-354bccb4-8989-44f0-bbda-421636b3e08a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:45.917768 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-354bccb4-8989-44f0-bbda-421636b3e08a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:45.918205 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-354bccb4-8989-44f0-bbda-421636b3e08a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:45.936115 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-510fb5cb-5e49-4489-afed-a2bc82440ec8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:45.936976 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-510fb5cb-5e49-4489-afed-a2bc82440ec8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 53545a7c-c683-49d6-941d-4445f2b09379] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:45.942813 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-510fb5cb-5e49-4489-afed-a2bc82440ec8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:45.943251 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-510fb5cb-5e49-4489-afed-a2bc82440ec8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:45.944003 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-510fb5cb-5e49-4489-afed-a2bc82440ec8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:45.944505 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-510fb5cb-5e49-4489-afed-a2bc82440ec8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:46.008903 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-510fb5cb-5e49-4489-afed-a2bc82440ec8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:46.016264 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-510fb5cb-5e49-4489-afed-a2bc82440ec8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379" status: 200 len: 1294 microversion: 2.1 time: 0.083901 Jul 19 11:08:46.017055 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 361/697] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:45 2022] GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379 => generated 1294 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:08:46.052601 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-354bccb4-8989-44f0-bbda-421636b3e08a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:46.060771 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-354bccb4-8989-44f0-bbda-421636b3e08a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb" status: 200 len: 1453 microversion: 2.1 time: 0.154952 Jul 19 11:08:46.061328 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 337/698] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:45 2022] GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb => generated 1453 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:47.034157 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-f961f854-8b9b-4c8e-a33f-f32b1e16e61f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:47.035024 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-f961f854-8b9b-4c8e-a33f-f32b1e16e61f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 53545a7c-c683-49d6-941d-4445f2b09379] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:47.042349 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f961f854-8b9b-4c8e-a33f-f32b1e16e61f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:47.042791 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f961f854-8b9b-4c8e-a33f-f32b1e16e61f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:47.043548 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f961f854-8b9b-4c8e-a33f-f32b1e16e61f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:47.044150 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f961f854-8b9b-4c8e-a33f-f32b1e16e61f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:47.305203 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-f961f854-8b9b-4c8e-a33f-f32b1e16e61f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:47.323196 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-f961f854-8b9b-4c8e-a33f-f32b1e16e61f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379" status: 200 len: 1368 microversion: 2.1 time: 0.293590 Jul 19 11:08:47.324563 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 362/699] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:47 2022] GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379 => generated 1368 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:48.027039 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-9a62c305-a954-4895-be2c-ddc046a3af0b req-e969976b-0d62-4a3e-b3be-0d313df56efb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "12cd8c55-b30f-40ee-8cf5-7c974b8caafb", "tag": "2648a82a-50bf-410f-86dd-82693a421316"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:08:48.035913 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-9a62c305-a954-4895-be2c-ddc046a3af0b req-e969976b-0d62-4a3e-b3be-0d313df56efb service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:48.036357 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-9a62c305-a954-4895-be2c-ddc046a3af0b req-e969976b-0d62-4a3e-b3be-0d313df56efb service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:48.047822 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-9a62c305-a954-4895-be2c-ddc046a3af0b req-e969976b-0d62-4a3e-b3be-0d313df56efb service nova] Creating event network-changed:2648a82a-50bf-410f-86dd-82693a421316 for instance 12cd8c55-b30f-40ee-8cf5-7c974b8caafb on hv-cinder-maste Jul 19 11:08:48.052897 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-9a62c305-a954-4895-be2c-ddc046a3af0b req-e969976b-0d62-4a3e-b3be-0d313df56efb service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029460 Jul 19 11:08:48.053859 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 338/700] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:08:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:08:48.281500 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-f8ba6f14-3dc2-43ee-ae8c-fa785210dd1a req-c8147ec2-b371-43c2-9b90-fef6a556baa2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "53545a7c-c683-49d6-941d-4445f2b09379", "tag": "bd4b190f-3901-45e6-89a0-d4cda5892170"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:08:48.299514 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-f8ba6f14-3dc2-43ee-ae8c-fa785210dd1a req-c8147ec2-b371-43c2-9b90-fef6a556baa2 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:48.299963 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-f8ba6f14-3dc2-43ee-ae8c-fa785210dd1a req-c8147ec2-b371-43c2-9b90-fef6a556baa2 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:48.312929 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-f8ba6f14-3dc2-43ee-ae8c-fa785210dd1a req-c8147ec2-b371-43c2-9b90-fef6a556baa2 service nova] Creating event network-changed:bd4b190f-3901-45e6-89a0-d4cda5892170 for instance 53545a7c-c683-49d6-941d-4445f2b09379 on hv-cinder-maste Jul 19 11:08:48.319155 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-f8ba6f14-3dc2-43ee-ae8c-fa785210dd1a req-c8147ec2-b371-43c2-9b90-fef6a556baa2 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041205 Jul 19 11:08:48.320874 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 363/701] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:08:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:08:48.342418 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-1e57212f-7dd8-4672-8cc6-36553986b832 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:48.343114 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-1e57212f-7dd8-4672-8cc6-36553986b832 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 53545a7c-c683-49d6-941d-4445f2b09379] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:48.349374 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1e57212f-7dd8-4672-8cc6-36553986b832 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:48.349827 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1e57212f-7dd8-4672-8cc6-36553986b832 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:48.350607 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1e57212f-7dd8-4672-8cc6-36553986b832 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:48.351095 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1e57212f-7dd8-4672-8cc6-36553986b832 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:48.602923 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-1e57212f-7dd8-4672-8cc6-36553986b832 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:48.610754 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-1e57212f-7dd8-4672-8cc6-36553986b832 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379" status: 200 len: 1368 microversion: 2.1 time: 0.272526 Jul 19 11:08:48.611532 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 339/702] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:48 2022] GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379 => generated 1368 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:49.629189 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-6b60d919-5e9c-470e-90f3-001bfd1a41bb tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:49.629883 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-6b60d919-5e9c-470e-90f3-001bfd1a41bb tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 53545a7c-c683-49d6-941d-4445f2b09379] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:49.635990 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6b60d919-5e9c-470e-90f3-001bfd1a41bb tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:49.636433 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6b60d919-5e9c-470e-90f3-001bfd1a41bb tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:49.637296 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6b60d919-5e9c-470e-90f3-001bfd1a41bb tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:49.637728 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6b60d919-5e9c-470e-90f3-001bfd1a41bb tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:49.768743 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-6b60d919-5e9c-470e-90f3-001bfd1a41bb tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:49.776748 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-6b60d919-5e9c-470e-90f3-001bfd1a41bb tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379" status: 200 len: 1368 microversion: 2.1 time: 0.152318 Jul 19 11:08:49.777320 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 364/703] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:49 2022] GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379 => generated 1368 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:50.793981 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-d0cc359b-62f4-45aa-a256-a65c9dab899e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:50.794577 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-d0cc359b-62f4-45aa-a256-a65c9dab899e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 53545a7c-c683-49d6-941d-4445f2b09379] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:50.800959 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d0cc359b-62f4-45aa-a256-a65c9dab899e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:50.801407 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d0cc359b-62f4-45aa-a256-a65c9dab899e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:50.802154 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d0cc359b-62f4-45aa-a256-a65c9dab899e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:50.802580 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d0cc359b-62f4-45aa-a256-a65c9dab899e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:50.932762 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-d0cc359b-62f4-45aa-a256-a65c9dab899e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:50.940145 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-d0cc359b-62f4-45aa-a256-a65c9dab899e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379" status: 200 len: 1368 microversion: 2.1 time: 0.150126 Jul 19 11:08:50.940718 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 340/704] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:50 2022] GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379 => generated 1368 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:51.957438 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-86cdf6ad-f8db-4804-b277-98e90dd82056 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:51.958040 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-86cdf6ad-f8db-4804-b277-98e90dd82056 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 53545a7c-c683-49d6-941d-4445f2b09379] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:51.964630 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-86cdf6ad-f8db-4804-b277-98e90dd82056 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:51.965103 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-86cdf6ad-f8db-4804-b277-98e90dd82056 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:51.965850 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-86cdf6ad-f8db-4804-b277-98e90dd82056 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:51.966278 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-86cdf6ad-f8db-4804-b277-98e90dd82056 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:52.098749 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-86cdf6ad-f8db-4804-b277-98e90dd82056 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:52.106666 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-86cdf6ad-f8db-4804-b277-98e90dd82056 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379" status: 200 len: 1368 microversion: 2.1 time: 0.153045 Jul 19 11:08:52.107227 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 365/705] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:51 2022] GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379 => generated 1368 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:52.821339 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-e8aadc6e-ef14-4590-b545-0c90dd2bffbb tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:08:52.822329 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-e8aadc6e-ef14-4590-b545-0c90dd2bffbb tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:52.828427 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e8aadc6e-ef14-4590-b545-0c90dd2bffbb tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:52.828936 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e8aadc6e-ef14-4590-b545-0c90dd2bffbb tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:52.829705 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e8aadc6e-ef14-4590-b545-0c90dd2bffbb tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:52.830149 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e8aadc6e-ef14-4590-b545-0c90dd2bffbb tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:52.852564 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.objects.instance [None req-e8aadc6e-ef14-4590-b545-0c90dd2bffbb tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lazy-loading 'flavor' on Instance uuid b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 {{(pid=117849) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 19 11:08:52.971055 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-e8aadc6e-ef14-4590-b545-0c90dd2bffbb tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "POST /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3/action" status: 200 len: 32 microversion: 2.1 time: 0.154254 Jul 19 11:08:52.971573 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 341/706] 10.222.0.33 () {60 vars in 1277 bytes} [Tue Jul 19 11:08:52 2022] POST /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3/action => generated 32 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 19 11:08:52.981109 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-2c66d544-bf93-4116-b9cd-501f4189e3b8 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:52.981717 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-2c66d544-bf93-4116-b9cd-501f4189e3b8 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:52.988404 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2c66d544-bf93-4116-b9cd-501f4189e3b8 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:52.988966 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2c66d544-bf93-4116-b9cd-501f4189e3b8 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:52.989828 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2c66d544-bf93-4116-b9cd-501f4189e3b8 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:52.990258 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2c66d544-bf93-4116-b9cd-501f4189e3b8 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:53.123266 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-2ecab6b7-163e-4014-9528-c809f2d7c2af tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:53.124524 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-2ecab6b7-163e-4014-9528-c809f2d7c2af tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 53545a7c-c683-49d6-941d-4445f2b09379] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:53.131008 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2ecab6b7-163e-4014-9528-c809f2d7c2af tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:53.131452 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2ecab6b7-163e-4014-9528-c809f2d7c2af tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:53.132211 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2ecab6b7-163e-4014-9528-c809f2d7c2af tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:53.132641 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2ecab6b7-163e-4014-9528-c809f2d7c2af tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:53.211961 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-2c66d544-bf93-4116-b9cd-501f4189e3b8 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:53.221416 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-2c66d544-bf93-4116-b9cd-501f4189e3b8 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1735 microversion: 2.1 time: 0.243942 Jul 19 11:08:53.221925 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 366/707] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:52 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1735 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:53.272370 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-2ecab6b7-163e-4014-9528-c809f2d7c2af tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:53.276399 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:08:53.281041 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-2ecab6b7-163e-4014-9528-c809f2d7c2af tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379" status: 200 len: 1368 microversion: 2.1 time: 0.161974 Jul 19 11:08:53.281617 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 342/708] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:53 2022] GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379 => generated 1368 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:08:53.743474 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-438e1b9f-ceb1-4916-abb6-6ff4e1816402 req-3c15344b-174b-43f4-bdbf-96173dd2041d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "53545a7c-c683-49d6-941d-4445f2b09379", "name": "network-vif-plugged", "status": "completed", "tag": "bd4b190f-3901-45e6-89a0-d4cda5892170"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:08:53.752119 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-438e1b9f-ceb1-4916-abb6-6ff4e1816402 req-3c15344b-174b-43f4-bdbf-96173dd2041d service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:53.752782 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-438e1b9f-ceb1-4916-abb6-6ff4e1816402 req-3c15344b-174b-43f4-bdbf-96173dd2041d service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:53.765200 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-438e1b9f-ceb1-4916-abb6-6ff4e1816402 req-3c15344b-174b-43f4-bdbf-96173dd2041d service nova] Creating event network-vif-plugged:bd4b190f-3901-45e6-89a0-d4cda5892170 for instance 53545a7c-c683-49d6-941d-4445f2b09379 on hv-cinder-maste Jul 19 11:08:53.769905 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-438e1b9f-ceb1-4916-abb6-6ff4e1816402 req-3c15344b-174b-43f4-bdbf-96173dd2041d service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030705 Jul 19 11:08:53.770646 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 367/709] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:08:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:08:54.238822 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-51f2009b-985d-4d4c-ad23-9c0287658560 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:54.239423 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-51f2009b-985d-4d4c-ad23-9c0287658560 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:54.242381 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell0:SELECT=2 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:08:54.243141 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell0:SELECT=10 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:08:54.251209 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-51f2009b-985d-4d4c-ad23-9c0287658560 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:54.251652 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-51f2009b-985d-4d4c-ad23-9c0287658560 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:54.252402 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-51f2009b-985d-4d4c-ad23-9c0287658560 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:54.252861 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-51f2009b-985d-4d4c-ad23-9c0287658560 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:54.294646 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell0:SELECT=2 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:08:54.299859 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-c8adb362-dfb3-421d-9130-7ef4f36f4205 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:54.301605 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-c8adb362-dfb3-421d-9130-7ef4f36f4205 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 53545a7c-c683-49d6-941d-4445f2b09379] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:54.309194 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c8adb362-dfb3-421d-9130-7ef4f36f4205 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:54.309982 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c8adb362-dfb3-421d-9130-7ef4f36f4205 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:54.310961 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c8adb362-dfb3-421d-9130-7ef4f36f4205 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:54.311590 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c8adb362-dfb3-421d-9130-7ef4f36f4205 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:54.390776 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-51f2009b-985d-4d4c-ad23-9c0287658560 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:54.400248 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-51f2009b-985d-4d4c-ad23-9c0287658560 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1735 microversion: 2.1 time: 0.165324 Jul 19 11:08:54.400841 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 343/710] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:54 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1735 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:54.451938 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-c8adb362-dfb3-421d-9130-7ef4f36f4205 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:54.459438 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-c8adb362-dfb3-421d-9130-7ef4f36f4205 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379" status: 200 len: 1368 microversion: 2.1 time: 0.165854 Jul 19 11:08:54.459998 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 368/711] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:54 2022] GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379 => generated 1368 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:08:55.421691 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-de7ecf9a-ccb9-409d-a6df-9e4a30adfabe tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:55.422293 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-de7ecf9a-ccb9-409d-a6df-9e4a30adfabe tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:55.429557 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-de7ecf9a-ccb9-409d-a6df-9e4a30adfabe tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:55.430001 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-de7ecf9a-ccb9-409d-a6df-9e4a30adfabe tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:55.430757 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-de7ecf9a-ccb9-409d-a6df-9e4a30adfabe tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:55.431262 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-de7ecf9a-ccb9-409d-a6df-9e4a30adfabe tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:55.475371 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-83fb48fb-7030-4283-9e00-cf65bf90756a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:55.476172 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-83fb48fb-7030-4283-9e00-cf65bf90756a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 53545a7c-c683-49d6-941d-4445f2b09379] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:55.482636 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-83fb48fb-7030-4283-9e00-cf65bf90756a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:55.483088 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-83fb48fb-7030-4283-9e00-cf65bf90756a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:55.483849 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-83fb48fb-7030-4283-9e00-cf65bf90756a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:55.484284 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-83fb48fb-7030-4283-9e00-cf65bf90756a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:55.561248 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-de7ecf9a-ccb9-409d-a6df-9e4a30adfabe tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:55.570271 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-de7ecf9a-ccb9-409d-a6df-9e4a30adfabe tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1735 microversion: 2.1 time: 0.151729 Jul 19 11:08:55.570841 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 344/712] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:55 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1735 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:55.617295 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-83fb48fb-7030-4283-9e00-cf65bf90756a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:55.624932 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-83fb48fb-7030-4283-9e00-cf65bf90756a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379" status: 200 len: 1368 microversion: 2.1 time: 0.153424 Jul 19 11:08:55.625508 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 369/713] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:55 2022] GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379 => generated 1368 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:08:56.588230 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-c90e8a2c-f7d6-421c-9ff3-a52118fb48d3 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:56.588867 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-c90e8a2c-f7d6-421c-9ff3-a52118fb48d3 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:56.595266 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c90e8a2c-f7d6-421c-9ff3-a52118fb48d3 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:56.595707 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c90e8a2c-f7d6-421c-9ff3-a52118fb48d3 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:56.596447 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c90e8a2c-f7d6-421c-9ff3-a52118fb48d3 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:56.596902 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c90e8a2c-f7d6-421c-9ff3-a52118fb48d3 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:56.642491 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-5f0a3c7c-87bb-44f3-b3f4-69325c9aabc1 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:56.643142 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-5f0a3c7c-87bb-44f3-b3f4-69325c9aabc1 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 53545a7c-c683-49d6-941d-4445f2b09379] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:56.650725 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5f0a3c7c-87bb-44f3-b3f4-69325c9aabc1 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:56.651174 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5f0a3c7c-87bb-44f3-b3f4-69325c9aabc1 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:56.651938 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5f0a3c7c-87bb-44f3-b3f4-69325c9aabc1 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:56.652369 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5f0a3c7c-87bb-44f3-b3f4-69325c9aabc1 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:56.731538 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-c90e8a2c-f7d6-421c-9ff3-a52118fb48d3 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:56.739501 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-c90e8a2c-f7d6-421c-9ff3-a52118fb48d3 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1735 microversion: 2.1 time: 0.155720 Jul 19 11:08:56.740062 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 345/714] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:56 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1735 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:56.792231 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-5f0a3c7c-87bb-44f3-b3f4-69325c9aabc1 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:56.799694 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-5f0a3c7c-87bb-44f3-b3f4-69325c9aabc1 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379" status: 200 len: 1558 microversion: 2.1 time: 0.161371 Jul 19 11:08:56.800251 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 370/715] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:56 2022] GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379 => generated 1558 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:08:56.815633 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-67574250-ddb4-46d9-9f3e-954d6ccdc13f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:56.816117 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-67574250-ddb4-46d9-9f3e-954d6ccdc13f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 53545a7c-c683-49d6-941d-4445f2b09379] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:56.822502 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-67574250-ddb4-46d9-9f3e-954d6ccdc13f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:56.822945 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-67574250-ddb4-46d9-9f3e-954d6ccdc13f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:56.823694 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-67574250-ddb4-46d9-9f3e-954d6ccdc13f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:56.824153 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-67574250-ddb4-46d9-9f3e-954d6ccdc13f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:56.904633 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-67574250-ddb4-46d9-9f3e-954d6ccdc13f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.092727 Jul 19 11:08:56.905248 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 346/716] 10.222.0.33 () {58 vars in 1274 bytes} [Tue Jul 19 11:08:56 2022] GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379/os-interface => generated 284 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:08:57.756231 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-a7d8d225-58d5-40e8-acef-254a2fd58500 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:57.756863 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-a7d8d225-58d5-40e8-acef-254a2fd58500 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:57.763562 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a7d8d225-58d5-40e8-acef-254a2fd58500 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:57.764009 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a7d8d225-58d5-40e8-acef-254a2fd58500 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:57.764849 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a7d8d225-58d5-40e8-acef-254a2fd58500 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:57.765213 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a7d8d225-58d5-40e8-acef-254a2fd58500 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:57.854686 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-8fcdd78e-f7c9-4b70-9b76-f4d1c317ae79 req-42335297-7c26-4241-a810-749f957bdc5f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "53545a7c-c683-49d6-941d-4445f2b09379", "tag": "bd4b190f-3901-45e6-89a0-d4cda5892170"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:08:57.869380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-8fcdd78e-f7c9-4b70-9b76-f4d1c317ae79 req-42335297-7c26-4241-a810-749f957bdc5f service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:57.870145 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-8fcdd78e-f7c9-4b70-9b76-f4d1c317ae79 req-42335297-7c26-4241-a810-749f957bdc5f service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:57.885975 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-8fcdd78e-f7c9-4b70-9b76-f4d1c317ae79 req-42335297-7c26-4241-a810-749f957bdc5f service nova] Creating event network-changed:bd4b190f-3901-45e6-89a0-d4cda5892170 for instance 53545a7c-c683-49d6-941d-4445f2b09379 on hv-cinder-maste Jul 19 11:08:57.891003 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-8fcdd78e-f7c9-4b70-9b76-f4d1c317ae79 req-42335297-7c26-4241-a810-749f957bdc5f service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040918 Jul 19 11:08:57.891594 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 347/717] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:08:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:08:57.919396 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-a7d8d225-58d5-40e8-acef-254a2fd58500 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:57.927485 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-a7d8d225-58d5-40e8-acef-254a2fd58500 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1735 microversion: 2.1 time: 0.175192 Jul 19 11:08:57.928323 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 371/718] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:57 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1735 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:08:58.945258 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-411e6fa4-c8ed-4818-9661-d07e6c45ba67 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:08:58.945948 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-411e6fa4-c8ed-4818-9661-d07e6c45ba67 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:08:58.952664 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-411e6fa4-c8ed-4818-9661-d07e6c45ba67 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:58.953153 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-411e6fa4-c8ed-4818-9661-d07e6c45ba67 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:58.953943 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-411e6fa4-c8ed-4818-9661-d07e6c45ba67 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:08:58.954378 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-411e6fa4-c8ed-4818-9661-d07e6c45ba67 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:08:59.081140 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-411e6fa4-c8ed-4818-9661-d07e6c45ba67 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:08:59.089457 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-411e6fa4-c8ed-4818-9661-d07e6c45ba67 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1735 microversion: 2.1 time: 0.148492 Jul 19 11:08:59.090013 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 348/719] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:08:58 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1735 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:00.106157 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-ea0fb7e6-10b8-4f4e-aefb-f5f4cfc0b19b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:00.106761 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-ea0fb7e6-10b8-4f4e-aefb-f5f4cfc0b19b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:00.113328 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ea0fb7e6-10b8-4f4e-aefb-f5f4cfc0b19b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:00.113779 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ea0fb7e6-10b8-4f4e-aefb-f5f4cfc0b19b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:00.114542 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ea0fb7e6-10b8-4f4e-aefb-f5f4cfc0b19b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:00.115014 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ea0fb7e6-10b8-4f4e-aefb-f5f4cfc0b19b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:00.245960 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-ea0fb7e6-10b8-4f4e-aefb-f5f4cfc0b19b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:00.254062 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-ea0fb7e6-10b8-4f4e-aefb-f5f4cfc0b19b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1735 microversion: 2.1 time: 0.151734 Jul 19 11:09:00.254629 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 372/720] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:00 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1735 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:01.270901 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-af9bc714-e402-41f4-8a64-c13c66274c98 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:01.271534 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-af9bc714-e402-41f4-8a64-c13c66274c98 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:01.278798 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-af9bc714-e402-41f4-8a64-c13c66274c98 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:01.279257 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-af9bc714-e402-41f4-8a64-c13c66274c98 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:01.280070 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-af9bc714-e402-41f4-8a64-c13c66274c98 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:01.280509 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-af9bc714-e402-41f4-8a64-c13c66274c98 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:01.411893 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-af9bc714-e402-41f4-8a64-c13c66274c98 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:01.419925 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-af9bc714-e402-41f4-8a64-c13c66274c98 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1735 microversion: 2.1 time: 0.152886 Jul 19 11:09:01.420487 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 349/721] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:01 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1735 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:02.437220 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-b2e0d3b0-c644-4a89-80ec-036af7f5726f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:02.437823 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-b2e0d3b0-c644-4a89-80ec-036af7f5726f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:02.444325 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b2e0d3b0-c644-4a89-80ec-036af7f5726f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:02.444806 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b2e0d3b0-c644-4a89-80ec-036af7f5726f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:02.445570 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b2e0d3b0-c644-4a89-80ec-036af7f5726f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:02.446610 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b2e0d3b0-c644-4a89-80ec-036af7f5726f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:02.573565 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-b2e0d3b0-c644-4a89-80ec-036af7f5726f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:02.581358 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-b2e0d3b0-c644-4a89-80ec-036af7f5726f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1735 microversion: 2.1 time: 0.148014 Jul 19 11:09:02.581930 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 373/722] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:02 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1735 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:03.598162 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-6faa5a88-381c-41f9-ac00-96b504ce0be0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:03.598753 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-6faa5a88-381c-41f9-ac00-96b504ce0be0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:03.605287 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6faa5a88-381c-41f9-ac00-96b504ce0be0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:03.605728 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6faa5a88-381c-41f9-ac00-96b504ce0be0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:03.606514 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6faa5a88-381c-41f9-ac00-96b504ce0be0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:03.606942 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6faa5a88-381c-41f9-ac00-96b504ce0be0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:03.740817 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-6faa5a88-381c-41f9-ac00-96b504ce0be0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:03.749324 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-6faa5a88-381c-41f9-ac00-96b504ce0be0 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1735 microversion: 2.1 time: 0.154747 Jul 19 11:09:03.749903 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 350/723] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:03 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1735 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:04.765635 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-2b07c501-13f0-4caa-959d-d4e21dc10846 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:04.766261 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-2b07c501-13f0-4caa-959d-d4e21dc10846 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:04.772766 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2b07c501-13f0-4caa-959d-d4e21dc10846 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:04.773211 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2b07c501-13f0-4caa-959d-d4e21dc10846 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:04.774609 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2b07c501-13f0-4caa-959d-d4e21dc10846 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:04.775038 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2b07c501-13f0-4caa-959d-d4e21dc10846 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:04.901241 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-2b07c501-13f0-4caa-959d-d4e21dc10846 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:04.909079 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-2b07c501-13f0-4caa-959d-d4e21dc10846 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1735 microversion: 2.1 time: 0.147294 Jul 19 11:09:04.909666 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 374/724] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:04 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1735 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:09:05.926196 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-394fec9d-b978-49f5-8790-6b3394d5bd56 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:05.926800 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-394fec9d-b978-49f5-8790-6b3394d5bd56 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:05.933531 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-394fec9d-b978-49f5-8790-6b3394d5bd56 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:05.933977 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-394fec9d-b978-49f5-8790-6b3394d5bd56 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:05.934839 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-394fec9d-b978-49f5-8790-6b3394d5bd56 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:05.935278 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-394fec9d-b978-49f5-8790-6b3394d5bd56 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:06.059409 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-394fec9d-b978-49f5-8790-6b3394d5bd56 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:06.067166 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-394fec9d-b978-49f5-8790-6b3394d5bd56 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1735 microversion: 2.1 time: 0.144809 Jul 19 11:09:06.067730 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 351/725] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:05 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1735 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:07.084137 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-ec469328-4dee-42f2-8d3a-7660f9be36f9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:07.084785 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-ec469328-4dee-42f2-8d3a-7660f9be36f9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:07.087081 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_api:SELECT=61,nova_api:DELETE=1 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:09:07.092357 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ec469328-4dee-42f2-8d3a-7660f9be36f9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:07.092826 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ec469328-4dee-42f2-8d3a-7660f9be36f9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:07.093584 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ec469328-4dee-42f2-8d3a-7660f9be36f9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:07.094016 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ec469328-4dee-42f2-8d3a-7660f9be36f9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:07.098832 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=100 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:09:07.225338 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-ec469328-4dee-42f2-8d3a-7660f9be36f9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:07.233679 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-ec469328-4dee-42f2-8d3a-7660f9be36f9 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1735 microversion: 2.1 time: 0.153381 Jul 19 11:09:07.234243 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 375/726] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:07 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1735 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:08.251005 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-e948c924-84f4-463d-8a53-3a97953c6001 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:08.251543 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-e948c924-84f4-463d-8a53-3a97953c6001 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:08.258032 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e948c924-84f4-463d-8a53-3a97953c6001 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:08.258481 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e948c924-84f4-463d-8a53-3a97953c6001 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:08.259234 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e948c924-84f4-463d-8a53-3a97953c6001 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:08.259659 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e948c924-84f4-463d-8a53-3a97953c6001 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:08.387971 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-e948c924-84f4-463d-8a53-3a97953c6001 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:08.395867 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-e948c924-84f4-463d-8a53-3a97953c6001 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1735 microversion: 2.1 time: 0.148996 Jul 19 11:09:08.396425 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 352/727] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:08 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1735 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:09.413101 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-358ca324-fb0b-4cb9-953b-49541f46021a tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:09.413731 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-358ca324-fb0b-4cb9-953b-49541f46021a tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:09.420750 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-358ca324-fb0b-4cb9-953b-49541f46021a tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:09.421191 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-358ca324-fb0b-4cb9-953b-49541f46021a tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:09.421937 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-358ca324-fb0b-4cb9-953b-49541f46021a tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:09.422364 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-358ca324-fb0b-4cb9-953b-49541f46021a tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:09.547809 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-358ca324-fb0b-4cb9-953b-49541f46021a tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:09.556335 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-358ca324-fb0b-4cb9-953b-49541f46021a tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1735 microversion: 2.1 time: 0.147040 Jul 19 11:09:09.556935 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 376/728] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:09 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1735 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:10.573683 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-701f1859-bd9d-43b8-94b5-d1e6f025ff08 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:10.574280 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-701f1859-bd9d-43b8-94b5-d1e6f025ff08 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:10.580712 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-701f1859-bd9d-43b8-94b5-d1e6f025ff08 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:10.581168 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-701f1859-bd9d-43b8-94b5-d1e6f025ff08 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:10.581918 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-701f1859-bd9d-43b8-94b5-d1e6f025ff08 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:10.582349 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-701f1859-bd9d-43b8-94b5-d1e6f025ff08 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:10.715947 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-701f1859-bd9d-43b8-94b5-d1e6f025ff08 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:10.723854 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-701f1859-bd9d-43b8-94b5-d1e6f025ff08 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1735 microversion: 2.1 time: 0.154589 Jul 19 11:09:10.724419 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 353/729] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:10 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1735 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:11.258639 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-3280729f-3c6a-4691-86e8-a31636ceb2df tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:09:11.259619 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-3280729f-3c6a-4691-86e8-a31636ceb2df tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:11.266141 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3280729f-3c6a-4691-86e8-a31636ceb2df tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:11.266590 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3280729f-3c6a-4691-86e8-a31636ceb2df tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:11.267349 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3280729f-3c6a-4691-86e8-a31636ceb2df tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:11.267786 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3280729f-3c6a-4691-86e8-a31636ceb2df tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:11.338705 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-3280729f-3c6a-4691-86e8-a31636ceb2df tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "POST /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a/action" status: 202 len: 0 microversion: 2.1 time: 0.083888 Jul 19 11:09:11.339724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 377/730] 10.222.0.33 () {60 vars in 1277 bytes} [Tue Jul 19 11:09:11 2022] POST /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a/action => generated 0 bytes in 85 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 19 11:09:11.347453 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-a82c254a-c02c-4920-b207-cab482ed68ee tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:11.348051 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-a82c254a-c02c-4920-b207-cab482ed68ee tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:11.354661 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a82c254a-c02c-4920-b207-cab482ed68ee tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:11.355101 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a82c254a-c02c-4920-b207-cab482ed68ee tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:11.355847 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a82c254a-c02c-4920-b207-cab482ed68ee tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:11.356276 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a82c254a-c02c-4920-b207-cab482ed68ee tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:11.498428 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-a82c254a-c02c-4920-b207-cab482ed68ee tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:11.506639 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-a82c254a-c02c-4920-b207-cab482ed68ee tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1752 microversion: 2.1 time: 0.162947 Jul 19 11:09:11.507195 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 354/731] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:11 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1752 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:09:11.742971 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-7ac35791-f9cc-469d-901a-add48374793c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:11.743663 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-7ac35791-f9cc-469d-901a-add48374793c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:11.750317 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7ac35791-f9cc-469d-901a-add48374793c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:11.752222 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7ac35791-f9cc-469d-901a-add48374793c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:11.752222 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7ac35791-f9cc-469d-901a-add48374793c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:11.752222 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7ac35791-f9cc-469d-901a-add48374793c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:11.878407 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-7ac35791-f9cc-469d-901a-add48374793c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:11.886964 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-7ac35791-f9cc-469d-901a-add48374793c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1735 microversion: 2.1 time: 0.148226 Jul 19 11:09:11.887517 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 378/732] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:11 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1735 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:12.291529 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-5f7c4a59-f5c9-40c3-b36b-690c7773ec41 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:12.292062 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-5f7c4a59-f5c9-40c3-b36b-690c7773ec41 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:12.294514 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_api:SELECT=66,nova_api:INSERT=4 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:09:12.300489 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5f7c4a59-f5c9-40c3-b36b-690c7773ec41 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:12.300662 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5f7c4a59-f5c9-40c3-b36b-690c7773ec41 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:12.301374 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5f7c4a59-f5c9-40c3-b36b-690c7773ec41 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:12.301811 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5f7c4a59-f5c9-40c3-b36b-690c7773ec41 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:12.427600 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.objects.instance [None req-5f7c4a59-f5c9-40c3-b36b-690c7773ec41 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lazy-loading 'services' on Instance uuid 940e8a5c-b365-40c2-8361-fbb677969487 {{(pid=117849) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 19 11:09:12.524288 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-0b1f79c9-69fd-4430-aa6f-3ccdc012f790 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:12.524937 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-0b1f79c9-69fd-4430-aa6f-3ccdc012f790 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:12.531416 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0b1f79c9-69fd-4430-aa6f-3ccdc012f790 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:12.531868 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0b1f79c9-69fd-4430-aa6f-3ccdc012f790 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:12.532633 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0b1f79c9-69fd-4430-aa6f-3ccdc012f790 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:12.533095 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0b1f79c9-69fd-4430-aa6f-3ccdc012f790 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:12.542973 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-5f7c4a59-f5c9-40c3-b36b-690c7773ec41 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "DELETE /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487/os-volume_attachments/1a8a01af-e70f-42d8-b1ae-24a6cd72175d" status: 202 len: 0 microversion: 2.1 time: 0.255721 Jul 19 11:09:12.543736 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 355/733] 10.222.0.33 () {58 vars in 1415 bytes} [Tue Jul 19 11:09:12 2022] DELETE /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487/os-volume_attachments/1a8a01af-e70f-42d8-b1ae-24a6cd72175d => generated 0 bytes in 257 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 19 11:09:12.682699 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-0b1f79c9-69fd-4430-aa6f-3ccdc012f790 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:12.690653 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-0b1f79c9-69fd-4430-aa6f-3ccdc012f790 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1752 microversion: 2.1 time: 0.170134 Jul 19 11:09:12.691224 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 379/734] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:12 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1752 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:09:12.904611 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-95de7362-c60f-46e1-8499-85081761b95e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:12.905253 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-95de7362-c60f-46e1-8499-85081761b95e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:12.911441 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-95de7362-c60f-46e1-8499-85081761b95e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:12.911892 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-95de7362-c60f-46e1-8499-85081761b95e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:12.912653 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-95de7362-c60f-46e1-8499-85081761b95e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:12.913119 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-95de7362-c60f-46e1-8499-85081761b95e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:13.041042 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-95de7362-c60f-46e1-8499-85081761b95e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:13.048997 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-95de7362-c60f-46e1-8499-85081761b95e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1715 microversion: 2.1 time: 0.148877 Jul 19 11:09:13.049557 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 356/735] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:12 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1715 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:13.149451 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-77df9149-d067-49a9-911c-6774dad39408 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:13.149946 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-77df9149-d067-49a9-911c-6774dad39408 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:13.156006 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-77df9149-d067-49a9-911c-6774dad39408 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:13.156379 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-77df9149-d067-49a9-911c-6774dad39408 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:13.157168 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-77df9149-d067-49a9-911c-6774dad39408 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:13.157608 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-77df9149-d067-49a9-911c-6774dad39408 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:13.239843 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.wsgi [None req-77df9149-d067-49a9-911c-6774dad39408 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] HTTP exception thrown: Cannot 'detach_volume' instance b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 while it is in vm_state rescued Jul 19 11:09:13.241506 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-77df9149-d067-49a9-911c-6774dad39408 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Request method failure captured: Jul 19 11:09:13.241506 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: request: DELETE /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3/os-volume_attachments/5ac37faf-6388-4d0e-9939-097bb5b52afc HTTP/1.1 Jul 19 11:09:13.241506 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept: application/json Jul 19 11:09:13.241506 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept-Encoding: identity Jul 19 11:09:13.241506 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Connection: close Jul 19 11:09:13.241506 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Length: 0 Jul 19 11:09:13.241506 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Type: application/json Jul 19 11:09:13.241506 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Host: 10.222.0.33 Jul 19 11:09:13.241506 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Openstack-System-Scope: None Jul 19 11:09:13.241506 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: User-Agent: python-urllib3/1.26.9 Jul 19 11:09:13.241506 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Auth-Token: gAAAAABi1o_D_huvBlD9ZJOIcq8Z1aOSnILmJp8MQgaQ8MuWsEAL_ZmZPsH--WyF20ahgvN08GveS7vk1bvyoVIPWYbHaxCp791BKfyC6yZ04rwPPH3vQlTPlGO54x_Ovlr-nuJTMqDQHYocrhpCc4nPGpje8DwiZIJCWR_YJdRwNKP8_4bwd5o Jul 19 11:09:13.241506 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Id: None Jul 19 11:09:13.241506 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Name: None Jul 19 11:09:13.241506 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Identity-Status: Confirmed Jul 19 11:09:13.241506 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Is-Admin-Project: True Jul 19 11:09:13.241506 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Id: default Jul 19 11:09:13.241506 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Name: Default Jul 19 11:09:13.241506 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Id: 05fceba2c70c40a399e71daabe8edef6 Jul 19 11:09:13.241506 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1654338802 Jul 19 11:09:13.241506 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Role: member,reader Jul 19 11:09:13.241506 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Roles: member,reader Jul 19 11:09:13.242994 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/05fceba2c70c40a399e71daabe8edef6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/05fceba2c70c40a399e71daabe8edef6"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/05fceba2c70c40a399e71daabe8edef6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_05fceba2c70c40a399e71daabe8edef6"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:09:13.242994 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1654338802 Jul 19 11:09:13.242994 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Id: 05fceba2c70c40a399e71daabe8edef6 Jul 19 11:09:13.242994 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1654338802 Jul 19 11:09:13.242994 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User: tempest-ServerRescueNegativeTestJSON-1654338802-project Jul 19 11:09:13.242994 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Id: default Jul 19 11:09:13.242994 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Name: Default Jul 19 11:09:13.242994 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Id: c95281b28eae450d8678583a929c3b8a Jul 19 11:09:13.242994 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1654338802-project Jul 19 11:09:13.242994 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: method: > Jul 19 11:09:13.242994 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: exception: Cannot 'detach_volume' instance b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 while it is in vm_state rescued Jul 19 11:09:13.243689 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: action_args: {'server_id': 'b9897cfb-9ec6-41f4-bcb5-ba1a978421d3', 'id': '5ac37faf-6388-4d0e-9939-097bb5b52afc'} Jul 19 11:09:13.243689 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:09:13.243689 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:09:13.243689 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 19 11:09:13.243689 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 19 11:09:13.243689 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 19 11:09:13.243689 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 19 11:09:13.243689 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Jul 19 11:09:13.243689 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Jul 19 11:09:13.243689 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 in vm_state rescued. Cannot detach_volume while the instance is in this state. Jul 19 11:09:13.243689 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:09:13.243689 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:09:13.243689 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:09:13.243689 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:09:13.243689 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:09:13.243689 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:09:13.243689 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:09:13.243689 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:09:13.243689 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:09:13.243689 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:09:13.244928 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Jul 19 11:09:13.244928 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Jul 19 11:09:13.244928 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Jul 19 11:09:13.244928 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Jul 19 11:09:13.244928 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 while it is in vm_state rescued Jul 19 11:09:13.244928 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:09:13.244928 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:09:13.244928 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:09:13.244928 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:09:13.244928 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:09:13.244928 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:09:13.244928 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:09:13.244928 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:09:13.244928 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 while it is in vm_state rescued Jul 19 11:09:13.244928 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:09:13.244928 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-77df9149-d067-49a9-911c-6774dad39408 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Returning 409 to user: Cannot 'detach_volume' instance b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 while it is in vm_state rescued {{(pid=117848) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:09:13.245893 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-77df9149-d067-49a9-911c-6774dad39408 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "DELETE /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3/os-volume_attachments/5ac37faf-6388-4d0e-9939-097bb5b52afc" status: 409 len: 152 microversion: 2.1 time: 0.098135 Jul 19 11:09:13.245893 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 380/736] 10.222.0.33 () {58 vars in 1415 bytes} [Tue Jul 19 11:09:13 2022] DELETE /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3/os-volume_attachments/5ac37faf-6388-4d0e-9939-097bb5b52afc => generated 152 bytes in 99 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Jul 19 11:09:13.252491 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-1749d288-7805-4a71-a97c-e370fa2f4b16 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:09:13.253161 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-1749d288-7805-4a71-a97c-e370fa2f4b16 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:13.260311 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1749d288-7805-4a71-a97c-e370fa2f4b16 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:13.260792 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1749d288-7805-4a71-a97c-e370fa2f4b16 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:13.261558 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1749d288-7805-4a71-a97c-e370fa2f4b16 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:13.261985 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1749d288-7805-4a71-a97c-e370fa2f4b16 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:13.335301 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-1749d288-7805-4a71-a97c-e370fa2f4b16 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "POST /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3/action" status: 202 len: 0 microversion: 2.1 time: 0.087112 Jul 19 11:09:13.336991 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 357/737] 10.222.0.33 () {60 vars in 1277 bytes} [Tue Jul 19 11:09:13 2022] POST /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3/action => generated 0 bytes in 90 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 19 11:09:13.343969 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-95d7cd01-7e82-487e-bcb3-87a92c07ac35 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:13.344807 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-95d7cd01-7e82-487e-bcb3-87a92c07ac35 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:13.354124 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-95d7cd01-7e82-487e-bcb3-87a92c07ac35 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:13.354815 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-95d7cd01-7e82-487e-bcb3-87a92c07ac35 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:13.355777 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-95d7cd01-7e82-487e-bcb3-87a92c07ac35 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:13.356385 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-95d7cd01-7e82-487e-bcb3-87a92c07ac35 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:13.512233 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-95d7cd01-7e82-487e-bcb3-87a92c07ac35 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:13.520125 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-95d7cd01-7e82-487e-bcb3-87a92c07ac35 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1723 microversion: 2.1 time: 0.180064 Jul 19 11:09:13.520685 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 381/738] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:13 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1723 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:13.711006 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-dd0db0c7-eaf8-44fc-a763-c4b5bcee423a tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:13.712020 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-dd0db0c7-eaf8-44fc-a763-c4b5bcee423a tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:13.720763 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dd0db0c7-eaf8-44fc-a763-c4b5bcee423a tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:13.721419 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dd0db0c7-eaf8-44fc-a763-c4b5bcee423a tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:13.722537 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dd0db0c7-eaf8-44fc-a763-c4b5bcee423a tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:13.723177 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dd0db0c7-eaf8-44fc-a763-c4b5bcee423a tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:13.869775 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-dd0db0c7-eaf8-44fc-a763-c4b5bcee423a tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:13.873285 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:09:13.879212 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-dd0db0c7-eaf8-44fc-a763-c4b5bcee423a tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1752 microversion: 2.1 time: 0.174823 Jul 19 11:09:13.879789 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 358/739] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:13 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1752 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:14.537473 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-70776dc8-e865-42ff-be73-9755e3bece2c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:14.538070 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-70776dc8-e865-42ff-be73-9755e3bece2c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:14.544473 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-70776dc8-e865-42ff-be73-9755e3bece2c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:14.544957 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-70776dc8-e865-42ff-be73-9755e3bece2c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:14.545715 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-70776dc8-e865-42ff-be73-9755e3bece2c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:14.546149 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-70776dc8-e865-42ff-be73-9755e3bece2c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:14.754707 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-70776dc8-e865-42ff-be73-9755e3bece2c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:14.763210 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-70776dc8-e865-42ff-be73-9755e3bece2c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1723 microversion: 2.1 time: 0.229652 Jul 19 11:09:14.763973 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 382/740] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:14 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1723 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:14.896016 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-7c126c77-5730-4348-bacf-9f55ffe308bd tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:14.896720 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-7c126c77-5730-4348-bacf-9f55ffe308bd tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:14.905074 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7c126c77-5730-4348-bacf-9f55ffe308bd tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:14.905528 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7c126c77-5730-4348-bacf-9f55ffe308bd tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:14.906293 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7c126c77-5730-4348-bacf-9f55ffe308bd tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:14.906860 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7c126c77-5730-4348-bacf-9f55ffe308bd tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:15.054751 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-7c126c77-5730-4348-bacf-9f55ffe308bd tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:15.063113 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-7c126c77-5730-4348-bacf-9f55ffe308bd tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1757 microversion: 2.1 time: 0.171544 Jul 19 11:09:15.063674 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 359/741] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:14 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1757 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:15.761432 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-302cd8e1-f0f6-4c29-b7a0-56565310018a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:09:15.767965 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-302cd8e1-f0f6-4c29-b7a0-56565310018a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:15.768414 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-302cd8e1-f0f6-4c29-b7a0-56565310018a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:15.780840 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-c08861e3-2e17-4be7-89cc-fb522fa4c428 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:15.781444 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-c08861e3-2e17-4be7-89cc-fb522fa4c428 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:15.787421 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c08861e3-2e17-4be7-89cc-fb522fa4c428 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:15.787872 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c08861e3-2e17-4be7-89cc-fb522fa4c428 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:15.788631 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c08861e3-2e17-4be7-89cc-fb522fa4c428 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:15.789094 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c08861e3-2e17-4be7-89cc-fb522fa4c428 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:15.790918 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-302cd8e1-f0f6-4c29-b7a0-56565310018a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Going to try to stop instance {{(pid=117848) force_stop /opt/stack/nova/nova/compute/api.py:2707}} Jul 19 11:09:15.834133 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-302cd8e1-f0f6-4c29-b7a0-56565310018a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "POST /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487/action" status: 202 len: 0 microversion: 2.1 time: 0.076270 Jul 19 11:09:15.834747 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 383/742] 10.222.0.33 () {60 vars in 1277 bytes} [Tue Jul 19 11:09:15 2022] POST /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487/action => generated 0 bytes in 78 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 19 11:09:15.843554 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-2e7d317a-1611-461c-857a-ba179c97170e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:15.844239 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-2e7d317a-1611-461c-857a-ba179c97170e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:15.850572 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2e7d317a-1611-461c-857a-ba179c97170e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:15.851021 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2e7d317a-1611-461c-857a-ba179c97170e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:15.851774 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2e7d317a-1611-461c-857a-ba179c97170e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:15.852237 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2e7d317a-1611-461c-857a-ba179c97170e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:15.920203 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-c08861e3-2e17-4be7-89cc-fb522fa4c428 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:15.928551 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-c08861e3-2e17-4be7-89cc-fb522fa4c428 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1723 microversion: 2.1 time: 0.151545 Jul 19 11:09:15.936294 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 360/743] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:15 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1723 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:09:15.983998 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-2e7d317a-1611-461c-857a-ba179c97170e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:15.991400 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-2e7d317a-1611-461c-857a-ba179c97170e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1677 microversion: 2.1 time: 0.151877 Jul 19 11:09:15.991952 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 384/744] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:15 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1677 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:09:16.080810 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-8f501c8a-9b88-4a57-909a-1e9c4d985856 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:16.081417 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-8f501c8a-9b88-4a57-909a-1e9c4d985856 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:16.088150 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8f501c8a-9b88-4a57-909a-1e9c4d985856 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:16.088598 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8f501c8a-9b88-4a57-909a-1e9c4d985856 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:16.089387 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8f501c8a-9b88-4a57-909a-1e9c4d985856 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:16.089824 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8f501c8a-9b88-4a57-909a-1e9c4d985856 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:16.230699 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-8f501c8a-9b88-4a57-909a-1e9c4d985856 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:16.238682 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-8f501c8a-9b88-4a57-909a-1e9c4d985856 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1757 microversion: 2.1 time: 0.161614 Jul 19 11:09:16.239178 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 361/745] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:16 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1757 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:16.944869 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-eb1c5e6a-7416-4f28-840d-4c14b39c601b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:16.945469 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-eb1c5e6a-7416-4f28-840d-4c14b39c601b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:16.951838 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-eb1c5e6a-7416-4f28-840d-4c14b39c601b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:16.952278 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-eb1c5e6a-7416-4f28-840d-4c14b39c601b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:16.953059 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-eb1c5e6a-7416-4f28-840d-4c14b39c601b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:16.953495 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-eb1c5e6a-7416-4f28-840d-4c14b39c601b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:17.008146 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-11cc02e8-1e95-4c23-b9d7-721d44825df0 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:17.008770 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-11cc02e8-1e95-4c23-b9d7-721d44825df0 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:17.015540 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-11cc02e8-1e95-4c23-b9d7-721d44825df0 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:17.016066 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-11cc02e8-1e95-4c23-b9d7-721d44825df0 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:17.016966 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-11cc02e8-1e95-4c23-b9d7-721d44825df0 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:17.017470 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-11cc02e8-1e95-4c23-b9d7-721d44825df0 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:17.085075 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-eb1c5e6a-7416-4f28-840d-4c14b39c601b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:17.095113 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-eb1c5e6a-7416-4f28-840d-4c14b39c601b tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1723 microversion: 2.1 time: 0.154054 Jul 19 11:09:17.095882 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 385/746] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:16 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1723 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:17.150632 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-11cc02e8-1e95-4c23-b9d7-721d44825df0 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:17.158073 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-11cc02e8-1e95-4c23-b9d7-721d44825df0 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1677 microversion: 2.1 time: 0.153816 Jul 19 11:09:17.158633 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 362/747] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:17 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1677 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:09:17.255627 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-75562b07-0252-4728-ad33-5d211ca90387 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:17.256160 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-75562b07-0252-4728-ad33-5d211ca90387 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:17.262406 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-75562b07-0252-4728-ad33-5d211ca90387 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:17.262854 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-75562b07-0252-4728-ad33-5d211ca90387 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:17.263618 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-75562b07-0252-4728-ad33-5d211ca90387 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:17.264055 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-75562b07-0252-4728-ad33-5d211ca90387 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:17.415161 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-75562b07-0252-4728-ad33-5d211ca90387 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:17.423751 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-75562b07-0252-4728-ad33-5d211ca90387 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1757 microversion: 2.1 time: 0.172070 Jul 19 11:09:17.424309 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 386/748] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:17 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1757 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:18.112940 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-157b3e0b-3e10-4b40-89ba-f22fe3418f3c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:18.113496 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-157b3e0b-3e10-4b40-89ba-f22fe3418f3c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:18.120434 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-157b3e0b-3e10-4b40-89ba-f22fe3418f3c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:18.120916 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-157b3e0b-3e10-4b40-89ba-f22fe3418f3c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:18.121672 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-157b3e0b-3e10-4b40-89ba-f22fe3418f3c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:18.122109 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-157b3e0b-3e10-4b40-89ba-f22fe3418f3c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:18.175188 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-8799b93b-b4dd-431b-b5ae-162ecc147883 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:18.175800 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-8799b93b-b4dd-431b-b5ae-162ecc147883 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:18.182866 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8799b93b-b4dd-431b-b5ae-162ecc147883 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:18.183314 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8799b93b-b4dd-431b-b5ae-162ecc147883 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:18.184078 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8799b93b-b4dd-431b-b5ae-162ecc147883 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:18.184509 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8799b93b-b4dd-431b-b5ae-162ecc147883 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:18.254378 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-157b3e0b-3e10-4b40-89ba-f22fe3418f3c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:18.262310 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-157b3e0b-3e10-4b40-89ba-f22fe3418f3c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1729 microversion: 2.1 time: 0.153325 Jul 19 11:09:18.262881 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 363/749] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:18 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1729 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:18.278161 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-305222d6-73ec-4b8a-a62c-cf73aa2f9f96 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:18.278648 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-305222d6-73ec-4b8a-a62c-cf73aa2f9f96 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:18.284602 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-305222d6-73ec-4b8a-a62c-cf73aa2f9f96 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:18.285071 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-305222d6-73ec-4b8a-a62c-cf73aa2f9f96 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:18.285823 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-305222d6-73ec-4b8a-a62c-cf73aa2f9f96 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:18.286251 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-305222d6-73ec-4b8a-a62c-cf73aa2f9f96 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:18.312767 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-8799b93b-b4dd-431b-b5ae-162ecc147883 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:18.320320 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-8799b93b-b4dd-431b-b5ae-162ecc147883 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1677 microversion: 2.1 time: 0.148904 Jul 19 11:09:18.320929 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 387/750] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:18 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1677 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:09:18.361755 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-305222d6-73ec-4b8a-a62c-cf73aa2f9f96 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.087251 Jul 19 11:09:18.362376 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 364/751] 10.222.0.33 () {58 vars in 1274 bytes} [Tue Jul 19 11:09:18 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3/os-interface => generated 284 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:09:18.440941 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-a26d5aa1-6c67-42eb-bb66-529f6aac7a90 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:18.441808 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-a26d5aa1-6c67-42eb-bb66-529f6aac7a90 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:18.448194 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a26d5aa1-6c67-42eb-bb66-529f6aac7a90 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:18.448645 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a26d5aa1-6c67-42eb-bb66-529f6aac7a90 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:18.449455 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a26d5aa1-6c67-42eb-bb66-529f6aac7a90 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:18.449895 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a26d5aa1-6c67-42eb-bb66-529f6aac7a90 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:18.597426 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-a26d5aa1-6c67-42eb-bb66-529f6aac7a90 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:18.606234 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-a26d5aa1-6c67-42eb-bb66-529f6aac7a90 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1757 microversion: 2.1 time: 0.169072 Jul 19 11:09:18.606808 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 388/752] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:18 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1757 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:19.247372 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-a7e2fff0-5b77-4299-9428-213e27433680 req-714780d0-9bb9-404a-81cb-f512011b2e9c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b9897cfb-9ec6-41f4-bcb5-ba1a978421d3", "tag": "bee94173-a17a-45b4-ab19-e862338cf16b"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:09:19.255723 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-a7e2fff0-5b77-4299-9428-213e27433680 req-714780d0-9bb9-404a-81cb-f512011b2e9c service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:19.258957 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-a7e2fff0-5b77-4299-9428-213e27433680 req-714780d0-9bb9-404a-81cb-f512011b2e9c service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:19.270711 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-a7e2fff0-5b77-4299-9428-213e27433680 req-714780d0-9bb9-404a-81cb-f512011b2e9c service nova] Creating event network-changed:bee94173-a17a-45b4-ab19-e862338cf16b for instance b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 on hv-cinder-maste Jul 19 11:09:19.276403 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-a7e2fff0-5b77-4299-9428-213e27433680 req-714780d0-9bb9-404a-81cb-f512011b2e9c service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033253 Jul 19 11:09:19.277205 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 365/753] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:09:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:09:19.337347 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-72c17cb9-8070-4392-bde9-21cc8ba5be47 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:19.337978 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-72c17cb9-8070-4392-bde9-21cc8ba5be47 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:19.344288 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-72c17cb9-8070-4392-bde9-21cc8ba5be47 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:19.344764 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-72c17cb9-8070-4392-bde9-21cc8ba5be47 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:19.345520 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-72c17cb9-8070-4392-bde9-21cc8ba5be47 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:19.345947 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-72c17cb9-8070-4392-bde9-21cc8ba5be47 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:19.561122 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-72c17cb9-8070-4392-bde9-21cc8ba5be47 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:19.565795 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:09:19.570352 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-72c17cb9-8070-4392-bde9-21cc8ba5be47 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1677 microversion: 2.1 time: 0.236766 Jul 19 11:09:19.571231 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 389/754] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:19 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1677 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:19.625203 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-a55190c5-e186-44b5-b727-d34453541f84 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:19.625904 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-a55190c5-e186-44b5-b727-d34453541f84 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:19.632805 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a55190c5-e186-44b5-b727-d34453541f84 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:19.634138 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a55190c5-e186-44b5-b727-d34453541f84 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:19.635642 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a55190c5-e186-44b5-b727-d34453541f84 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:19.636616 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a55190c5-e186-44b5-b727-d34453541f84 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:19.791007 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-a55190c5-e186-44b5-b727-d34453541f84 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:19.799543 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-a55190c5-e186-44b5-b727-d34453541f84 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1757 microversion: 2.1 time: 0.179080 Jul 19 11:09:19.800120 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 366/755] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:19 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1757 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:20.588176 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-c870571a-e7cf-4eb8-8a93-f17abb35eb59 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:20.588815 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-c870571a-e7cf-4eb8-8a93-f17abb35eb59 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:20.596256 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c870571a-e7cf-4eb8-8a93-f17abb35eb59 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:20.596729 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c870571a-e7cf-4eb8-8a93-f17abb35eb59 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:20.597481 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c870571a-e7cf-4eb8-8a93-f17abb35eb59 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:20.597915 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c870571a-e7cf-4eb8-8a93-f17abb35eb59 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:20.727151 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-c870571a-e7cf-4eb8-8a93-f17abb35eb59 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:20.734819 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-c870571a-e7cf-4eb8-8a93-f17abb35eb59 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1677 microversion: 2.1 time: 0.151524 Jul 19 11:09:20.735421 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 390/756] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:20 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1677 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:20.817152 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-e8820447-6af4-4050-86fb-8c3f24b6ee39 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:20.817754 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-e8820447-6af4-4050-86fb-8c3f24b6ee39 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:20.824845 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e8820447-6af4-4050-86fb-8c3f24b6ee39 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:20.825302 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e8820447-6af4-4050-86fb-8c3f24b6ee39 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:20.826058 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e8820447-6af4-4050-86fb-8c3f24b6ee39 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:20.826493 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e8820447-6af4-4050-86fb-8c3f24b6ee39 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:20.968497 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-e8820447-6af4-4050-86fb-8c3f24b6ee39 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:20.976355 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-e8820447-6af4-4050-86fb-8c3f24b6ee39 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1757 microversion: 2.1 time: 0.163054 Jul 19 11:09:20.976945 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 367/757] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:20 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1757 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:21.290498 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-45b105c6-039b-48e9-8442-ee59e27a0a38 req-d8816f65-e5dd-4528-bed2-79209c9fdfb4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b9897cfb-9ec6-41f4-bcb5-ba1a978421d3", "tag": "bee94173-a17a-45b4-ab19-e862338cf16b"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:09:21.298565 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-45b105c6-039b-48e9-8442-ee59e27a0a38 req-d8816f65-e5dd-4528-bed2-79209c9fdfb4 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:21.299002 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-45b105c6-039b-48e9-8442-ee59e27a0a38 req-d8816f65-e5dd-4528-bed2-79209c9fdfb4 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:21.309361 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-45b105c6-039b-48e9-8442-ee59e27a0a38 req-d8816f65-e5dd-4528-bed2-79209c9fdfb4 service nova] Creating event network-changed:bee94173-a17a-45b4-ab19-e862338cf16b for instance b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 on hv-cinder-maste Jul 19 11:09:21.313986 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-45b105c6-039b-48e9-8442-ee59e27a0a38 req-d8816f65-e5dd-4528-bed2-79209c9fdfb4 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.027220 Jul 19 11:09:21.314530 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 391/758] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:09:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:09:21.752180 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-817f61cb-b331-4f06-8d43-1a4eedc3b2e7 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:21.752830 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-817f61cb-b331-4f06-8d43-1a4eedc3b2e7 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:21.758861 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-817f61cb-b331-4f06-8d43-1a4eedc3b2e7 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:21.760132 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-817f61cb-b331-4f06-8d43-1a4eedc3b2e7 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:21.760826 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-817f61cb-b331-4f06-8d43-1a4eedc3b2e7 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:21.761267 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-817f61cb-b331-4f06-8d43-1a4eedc3b2e7 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:21.894642 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-817f61cb-b331-4f06-8d43-1a4eedc3b2e7 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:21.902487 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-817f61cb-b331-4f06-8d43-1a4eedc3b2e7 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1677 microversion: 2.1 time: 0.154455 Jul 19 11:09:21.903053 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 368/759] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:21 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1677 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:21.995181 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-7f32946b-6dc4-45fe-8940-801ef0535c8d tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:21.996017 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-7f32946b-6dc4-45fe-8940-801ef0535c8d tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:22.003106 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7f32946b-6dc4-45fe-8940-801ef0535c8d tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:22.003554 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7f32946b-6dc4-45fe-8940-801ef0535c8d tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:22.004313 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7f32946b-6dc4-45fe-8940-801ef0535c8d tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:22.004776 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7f32946b-6dc4-45fe-8940-801ef0535c8d tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:22.150914 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-7f32946b-6dc4-45fe-8940-801ef0535c8d tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:22.158926 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-7f32946b-6dc4-45fe-8940-801ef0535c8d tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1757 microversion: 2.1 time: 0.169103 Jul 19 11:09:22.159498 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 392/760] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:21 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1757 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:22.919529 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-6a6f267a-65ad-4451-aee3-0d1e04f3b127 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:22.920129 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-6a6f267a-65ad-4451-aee3-0d1e04f3b127 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:22.930264 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6a6f267a-65ad-4451-aee3-0d1e04f3b127 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:22.930943 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6a6f267a-65ad-4451-aee3-0d1e04f3b127 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:22.932059 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6a6f267a-65ad-4451-aee3-0d1e04f3b127 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:22.932492 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6a6f267a-65ad-4451-aee3-0d1e04f3b127 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:22.961053 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-44ca7f84-5ad7-43ac-967a-ec9ccb9934a9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:22.961560 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-44ca7f84-5ad7-43ac-967a-ec9ccb9934a9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:22.969116 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-44ca7f84-5ad7-43ac-967a-ec9ccb9934a9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:22.969560 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-44ca7f84-5ad7-43ac-967a-ec9ccb9934a9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:22.970317 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-44ca7f84-5ad7-43ac-967a-ec9ccb9934a9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:22.970745 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-44ca7f84-5ad7-43ac-967a-ec9ccb9934a9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:22.999616 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-44ca7f84-5ad7-43ac-967a-ec9ccb9934a9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326/os-volume_attachments/66afeab5-b4b2-42fc-8273-20e9cfb6bbe8" status: 200 len: 194 microversion: 2.20 time: 0.042488 Jul 19 11:09:23.000287 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 393/761] 10.222.0.33 () {60 vars in 1453 bytes} [Tue Jul 19 11:09:22 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326/os-volume_attachments/66afeab5-b4b2-42fc-8273-20e9cfb6bbe8 => generated 194 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 19 11:09:23.063415 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-0db3a94b-57d8-4634-b007-2cba1cdfd5cd tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:23.063606 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-6a6f267a-65ad-4451-aee3-0d1e04f3b127 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:23.063913 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-0db3a94b-57d8-4634-b007-2cba1cdfd5cd tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:23.070147 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0db3a94b-57d8-4634-b007-2cba1cdfd5cd tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:23.070598 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0db3a94b-57d8-4634-b007-2cba1cdfd5cd tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:23.071373 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0db3a94b-57d8-4634-b007-2cba1cdfd5cd tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:23.071473 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-6a6f267a-65ad-4451-aee3-0d1e04f3b127 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1677 microversion: 2.1 time: 0.155898 Jul 19 11:09:23.072432 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0db3a94b-57d8-4634-b007-2cba1cdfd5cd tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:23.072762 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 369/762] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:22 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1677 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:23.151086 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.objects.instance [None req-0db3a94b-57d8-4634-b007-2cba1cdfd5cd tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lazy-loading 'services' on Instance uuid f52fbfae-577b-403d-a4d3-9a4b79ae2326 {{(pid=117848) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 19 11:09:23.175184 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-f7251540-5f88-42b5-a277-d67da2d51bcc tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:23.175862 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-f7251540-5f88-42b5-a277-d67da2d51bcc tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:23.182462 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f7251540-5f88-42b5-a277-d67da2d51bcc tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:23.182976 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f7251540-5f88-42b5-a277-d67da2d51bcc tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:23.183792 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f7251540-5f88-42b5-a277-d67da2d51bcc tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:23.184304 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f7251540-5f88-42b5-a277-d67da2d51bcc tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:23.222305 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-0db3a94b-57d8-4634-b007-2cba1cdfd5cd tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "DELETE /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326/os-volume_attachments/66afeab5-b4b2-42fc-8273-20e9cfb6bbe8" status: 202 len: 0 microversion: 2.20 time: 0.162721 Jul 19 11:09:23.222971 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 394/763] 10.222.0.33 () {60 vars in 1456 bytes} [Tue Jul 19 11:09:23 2022] DELETE /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326/os-volume_attachments/66afeab5-b4b2-42fc-8273-20e9cfb6bbe8 => generated 0 bytes in 164 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Jul 19 11:09:23.232972 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-f6e3254f-8bd1-46b0-b462-3c785ec2c76a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:23.234123 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-f6e3254f-8bd1-46b0-b462-3c785ec2c76a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:23.241261 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f6e3254f-8bd1-46b0-b462-3c785ec2c76a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:23.241772 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f6e3254f-8bd1-46b0-b462-3c785ec2c76a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:23.242709 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f6e3254f-8bd1-46b0-b462-3c785ec2c76a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:23.243252 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f6e3254f-8bd1-46b0-b462-3c785ec2c76a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:23.281481 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-f6e3254f-8bd1-46b0-b462-3c785ec2c76a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053820 Jul 19 11:09:23.282243 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 395/764] 10.222.0.33 () {60 vars in 1342 bytes} [Tue Jul 19 11:09:23 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 19 11:09:23.331455 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-f7251540-5f88-42b5-a277-d67da2d51bcc tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:23.339623 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-f7251540-5f88-42b5-a277-d67da2d51bcc tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1757 microversion: 2.1 time: 0.168379 Jul 19 11:09:23.340191 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 370/765] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:23 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1757 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:09:24.089367 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-8edca59d-2eff-42cd-9528-bf2baa8174a9 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:24.089966 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-8edca59d-2eff-42cd-9528-bf2baa8174a9 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:24.096324 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8edca59d-2eff-42cd-9528-bf2baa8174a9 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:24.096800 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8edca59d-2eff-42cd-9528-bf2baa8174a9 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:24.097557 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8edca59d-2eff-42cd-9528-bf2baa8174a9 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:24.097985 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8edca59d-2eff-42cd-9528-bf2baa8174a9 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:24.284957 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-8edca59d-2eff-42cd-9528-bf2baa8174a9 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:24.292725 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-8edca59d-2eff-42cd-9528-bf2baa8174a9 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1677 microversion: 2.1 time: 0.207771 Jul 19 11:09:24.293383 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 396/766] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:24 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1677 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:24.293515 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-46959613-3214-42b9-8e25-958ffa1c6b26 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:24.294446 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-46959613-3214-42b9-8e25-958ffa1c6b26 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:24.300912 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-46959613-3214-42b9-8e25-958ffa1c6b26 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:24.301357 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-46959613-3214-42b9-8e25-958ffa1c6b26 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:24.302105 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-46959613-3214-42b9-8e25-958ffa1c6b26 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:24.302537 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-46959613-3214-42b9-8e25-958ffa1c6b26 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:24.332887 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-46959613-3214-42b9-8e25-958ffa1c6b26 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043413 Jul 19 11:09:24.333450 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 371/767] 10.222.0.33 () {60 vars in 1342 bytes} [Tue Jul 19 11:09:24 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 19 11:09:24.356620 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-24b772d6-eaad-4a3b-9f01-1da39dcffbc6 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:24.357260 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-24b772d6-eaad-4a3b-9f01-1da39dcffbc6 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:24.363370 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-24b772d6-eaad-4a3b-9f01-1da39dcffbc6 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:24.363816 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-24b772d6-eaad-4a3b-9f01-1da39dcffbc6 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:24.364574 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-24b772d6-eaad-4a3b-9f01-1da39dcffbc6 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:24.365036 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-24b772d6-eaad-4a3b-9f01-1da39dcffbc6 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:24.513598 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-24b772d6-eaad-4a3b-9f01-1da39dcffbc6 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:24.521495 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-24b772d6-eaad-4a3b-9f01-1da39dcffbc6 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1757 microversion: 2.1 time: 0.168653 Jul 19 11:09:24.522073 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 397/768] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:24 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1757 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:25.309981 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-cbc32c42-6d5d-44ce-97e0-07c4a5f98089 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:25.310541 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-cbc32c42-6d5d-44ce-97e0-07c4a5f98089 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:25.316946 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cbc32c42-6d5d-44ce-97e0-07c4a5f98089 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:25.317395 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cbc32c42-6d5d-44ce-97e0-07c4a5f98089 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:25.318148 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cbc32c42-6d5d-44ce-97e0-07c4a5f98089 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:25.318585 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cbc32c42-6d5d-44ce-97e0-07c4a5f98089 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:25.344631 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-38ac05a8-6447-42c1-8189-eef52b6b073d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:25.345646 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-38ac05a8-6447-42c1-8189-eef52b6b073d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:25.352147 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-38ac05a8-6447-42c1-8189-eef52b6b073d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:25.352590 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-38ac05a8-6447-42c1-8189-eef52b6b073d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:25.354065 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-38ac05a8-6447-42c1-8189-eef52b6b073d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:25.354603 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-38ac05a8-6447-42c1-8189-eef52b6b073d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:25.383075 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-38ac05a8-6447-42c1-8189-eef52b6b073d tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.042414 Jul 19 11:09:25.383634 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 398/769] 10.222.0.33 () {60 vars in 1342 bytes} [Tue Jul 19 11:09:25 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326/os-volume_attachments => generated 25 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:09:25.443165 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-728e0f23-d239-45b5-9352-3ae2fbacbdbf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:25.443659 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-728e0f23-d239-45b5-9352-3ae2fbacbdbf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:25.449788 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-728e0f23-d239-45b5-9352-3ae2fbacbdbf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:25.450236 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-728e0f23-d239-45b5-9352-3ae2fbacbdbf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:25.450998 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-728e0f23-d239-45b5-9352-3ae2fbacbdbf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:25.451430 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-728e0f23-d239-45b5-9352-3ae2fbacbdbf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:25.476137 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-728e0f23-d239-45b5-9352-3ae2fbacbdbf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Going to try to terminate instance {{(pid=117848) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 19 11:09:25.502426 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-cbc32c42-6d5d-44ce-97e0-07c4a5f98089 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:25.509947 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-cbc32c42-6d5d-44ce-97e0-07c4a5f98089 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1677 microversion: 2.1 time: 0.203897 Jul 19 11:09:25.510506 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 372/770] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:25 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1677 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:25.539417 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-2dbdb6ea-b59a-4978-bb7c-d137254d442f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:25.540050 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-2dbdb6ea-b59a-4978-bb7c-d137254d442f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:25.547320 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2dbdb6ea-b59a-4978-bb7c-d137254d442f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:25.547769 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2dbdb6ea-b59a-4978-bb7c-d137254d442f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:25.548671 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2dbdb6ea-b59a-4978-bb7c-d137254d442f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:25.549525 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2dbdb6ea-b59a-4978-bb7c-d137254d442f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:25.563763 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-728e0f23-d239-45b5-9352-3ae2fbacbdbf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "DELETE /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 204 len: 0 microversion: 2.20 time: 0.123803 Jul 19 11:09:25.564986 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 399/771] 10.222.0.33 () {60 vars in 1279 bytes} [Tue Jul 19 11:09:25 2022] DELETE /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Jul 19 11:09:25.573906 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-746c8737-155d-405b-a27c-722a28e32475 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:25.574592 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-746c8737-155d-405b-a27c-722a28e32475 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:25.580442 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-746c8737-155d-405b-a27c-722a28e32475 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:25.580923 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-746c8737-155d-405b-a27c-722a28e32475 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:25.581675 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-746c8737-155d-405b-a27c-722a28e32475 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:25.582102 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-746c8737-155d-405b-a27c-722a28e32475 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:25.691432 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-2dbdb6ea-b59a-4978-bb7c-d137254d442f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:25.700114 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-2dbdb6ea-b59a-4978-bb7c-d137254d442f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1757 microversion: 2.1 time: 0.164509 Jul 19 11:09:25.700688 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 373/772] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:25 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1757 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:09:25.713318 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-746c8737-155d-405b-a27c-722a28e32475 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:25.721362 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-746c8737-155d-405b-a27c-722a28e32475 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:25.722885 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-746c8737-155d-405b-a27c-722a28e32475 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:25.724039 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-746c8737-155d-405b-a27c-722a28e32475 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1722 microversion: 2.20 time: 0.154247 Jul 19 11:09:25.724594 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 400/773] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:09:25 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1722 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:09:26.530698 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-971c42a0-9c0a-4254-b3bf-d66cdb1f1fdf tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:26.531296 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-971c42a0-9c0a-4254-b3bf-d66cdb1f1fdf tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:26.538084 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-971c42a0-9c0a-4254-b3bf-d66cdb1f1fdf tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:26.538536 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-971c42a0-9c0a-4254-b3bf-d66cdb1f1fdf tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:26.539298 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-971c42a0-9c0a-4254-b3bf-d66cdb1f1fdf tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:26.539734 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-971c42a0-9c0a-4254-b3bf-d66cdb1f1fdf tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:26.664201 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-971c42a0-9c0a-4254-b3bf-d66cdb1f1fdf tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:26.671620 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-971c42a0-9c0a-4254-b3bf-d66cdb1f1fdf tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1677 microversion: 2.1 time: 0.144781 Jul 19 11:09:26.672178 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 374/774] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:26 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1677 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:26.717019 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-744ba51a-54f8-4c77-aef0-bf869001334f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:26.717681 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-744ba51a-54f8-4c77-aef0-bf869001334f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:26.724113 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-744ba51a-54f8-4c77-aef0-bf869001334f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:26.724561 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-744ba51a-54f8-4c77-aef0-bf869001334f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:26.725374 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-744ba51a-54f8-4c77-aef0-bf869001334f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:26.725790 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-744ba51a-54f8-4c77-aef0-bf869001334f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:26.742141 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-7a861407-567a-4c21-8dfa-c7554d0c0695 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:26.742962 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-7a861407-567a-4c21-8dfa-c7554d0c0695 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:26.749823 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7a861407-567a-4c21-8dfa-c7554d0c0695 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:26.750274 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7a861407-567a-4c21-8dfa-c7554d0c0695 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:26.751039 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7a861407-567a-4c21-8dfa-c7554d0c0695 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:26.751475 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7a861407-567a-4c21-8dfa-c7554d0c0695 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:26.875641 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-744ba51a-54f8-4c77-aef0-bf869001334f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:26.883665 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-7a861407-567a-4c21-8dfa-c7554d0c0695 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:26.883916 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-744ba51a-54f8-4c77-aef0-bf869001334f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1757 microversion: 2.1 time: 0.170556 Jul 19 11:09:26.884383 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 401/775] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:26 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1757 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:26.891582 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-7a861407-567a-4c21-8dfa-c7554d0c0695 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:26.893456 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-7a861407-567a-4c21-8dfa-c7554d0c0695 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:26.894592 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-7a861407-567a-4c21-8dfa-c7554d0c0695 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1722 microversion: 2.20 time: 0.156275 Jul 19 11:09:26.895132 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 375/776] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:09:26 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1722 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:09:27.688711 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-ab68a47a-290c-46b1-bbd8-cc8a13738416 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:27.689336 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-ab68a47a-290c-46b1-bbd8-cc8a13738416 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:27.695600 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ab68a47a-290c-46b1-bbd8-cc8a13738416 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:27.696045 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ab68a47a-290c-46b1-bbd8-cc8a13738416 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:27.696835 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ab68a47a-290c-46b1-bbd8-cc8a13738416 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:27.697267 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ab68a47a-290c-46b1-bbd8-cc8a13738416 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:27.897350 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-ab68a47a-290c-46b1-bbd8-cc8a13738416 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:27.901566 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-bf512d99-32f8-4350-884b-2abf5f4fc27b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:27.902174 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-bf512d99-32f8-4350-884b-2abf5f4fc27b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:27.906016 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-ab68a47a-290c-46b1-bbd8-cc8a13738416 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1677 microversion: 2.1 time: 0.220899 Jul 19 11:09:27.906629 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 402/777] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:27 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1677 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:27.911297 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bf512d99-32f8-4350-884b-2abf5f4fc27b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:27.912066 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bf512d99-32f8-4350-884b-2abf5f4fc27b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:27.913177 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bf512d99-32f8-4350-884b-2abf5f4fc27b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:27.913961 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bf512d99-32f8-4350-884b-2abf5f4fc27b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:27.916733 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-01fd4bcc-cafd-42e7-b3eb-55c62058fd67 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:27.916733 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-01fd4bcc-cafd-42e7-b3eb-55c62058fd67 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:27.923290 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-01fd4bcc-cafd-42e7-b3eb-55c62058fd67 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:27.923773 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-01fd4bcc-cafd-42e7-b3eb-55c62058fd67 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:27.924553 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-01fd4bcc-cafd-42e7-b3eb-55c62058fd67 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:27.926706 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-01fd4bcc-cafd-42e7-b3eb-55c62058fd67 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:27.942309 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=100 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:09:28.072064 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-bf512d99-32f8-4350-884b-2abf5f4fc27b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:28.081153 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-bf512d99-32f8-4350-884b-2abf5f4fc27b tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1757 microversion: 2.1 time: 0.183958 Jul 19 11:09:28.081883 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 376/778] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:27 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1757 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:09:28.172873 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-01fd4bcc-cafd-42e7-b3eb-55c62058fd67 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:28.181665 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-01fd4bcc-cafd-42e7-b3eb-55c62058fd67 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:28.183189 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-01fd4bcc-cafd-42e7-b3eb-55c62058fd67 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:28.184350 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-01fd4bcc-cafd-42e7-b3eb-55c62058fd67 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 200 len: 1722 microversion: 2.20 time: 0.272647 Jul 19 11:09:28.184949 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 403/779] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:09:27 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 1722 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:09:28.715435 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-069e4b34-fd89-426f-bbdc-299d73e12d81 req-62ddb3b1-39a1-4726-86c4-a2002cfceea0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f52fbfae-577b-403d-a4d3-9a4b79ae2326", "name": "network-vif-deleted", "tag": "748a7315-79e9-4ab8-ad6a-336c057e8b2c"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:09:28.723596 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-069e4b34-fd89-426f-bbdc-299d73e12d81 req-62ddb3b1-39a1-4726-86c4-a2002cfceea0 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:28.723969 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-069e4b34-fd89-426f-bbdc-299d73e12d81 req-62ddb3b1-39a1-4726-86c4-a2002cfceea0 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:28.735015 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-069e4b34-fd89-426f-bbdc-299d73e12d81 req-62ddb3b1-39a1-4726-86c4-a2002cfceea0 service nova] Creating event network-vif-deleted:748a7315-79e9-4ab8-ad6a-336c057e8b2c for instance f52fbfae-577b-403d-a4d3-9a4b79ae2326 on hv-cinder-maste Jul 19 11:09:28.739935 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-069e4b34-fd89-426f-bbdc-299d73e12d81 req-62ddb3b1-39a1-4726-86c4-a2002cfceea0 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.028125 Jul 19 11:09:28.740730 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 377/780] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:09:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:09:28.924951 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-88cf4178-fa7a-4b45-97db-028320626962 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:28.926369 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-88cf4178-fa7a-4b45-97db-028320626962 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:28.937332 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-88cf4178-fa7a-4b45-97db-028320626962 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:28.937886 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-88cf4178-fa7a-4b45-97db-028320626962 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:28.939466 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-88cf4178-fa7a-4b45-97db-028320626962 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:28.940646 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-88cf4178-fa7a-4b45-97db-028320626962 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:29.100875 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-1e01f57f-4783-4c10-bac8-b3209139d550 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:29.101613 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-1e01f57f-4783-4c10-bac8-b3209139d550 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:29.109174 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1e01f57f-4783-4c10-bac8-b3209139d550 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:29.109609 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1e01f57f-4783-4c10-bac8-b3209139d550 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:29.110670 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1e01f57f-4783-4c10-bac8-b3209139d550 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:29.111104 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1e01f57f-4783-4c10-bac8-b3209139d550 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:29.268425 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-1e01f57f-4783-4c10-bac8-b3209139d550 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:29.277717 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-1e01f57f-4783-4c10-bac8-b3209139d550 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1757 microversion: 2.1 time: 0.182065 Jul 19 11:09:29.278302 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 378/781] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:29 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1757 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:09:29.283344 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-90cb6320-22cb-4b64-9aa4-2242061c156f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:29.284332 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-90cb6320-22cb-4b64-9aa4-2242061c156f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:29.292293 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-90cb6320-22cb-4b64-9aa4-2242061c156f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:29.293068 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-90cb6320-22cb-4b64-9aa4-2242061c156f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:29.294394 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-90cb6320-22cb-4b64-9aa4-2242061c156f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:29.294830 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-90cb6320-22cb-4b64-9aa4-2242061c156f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:29.309832 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.wsgi [None req-90cb6320-22cb-4b64-9aa4-2242061c156f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] HTTP exception thrown: Instance f52fbfae-577b-403d-a4d3-9a4b79ae2326 could not be found. Jul 19 11:09:29.311665 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-90cb6320-22cb-4b64-9aa4-2242061c156f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Request method failure captured: Jul 19 11:09:29.311665 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: request: GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 HTTP/1.1 Jul 19 11:09:29.311665 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept: application/json Jul 19 11:09:29.311665 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept-Encoding: identity Jul 19 11:09:29.311665 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Connection: close Jul 19 11:09:29.311665 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Length: 0 Jul 19 11:09:29.311665 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Type: application/json Jul 19 11:09:29.311665 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Host: 10.222.0.33 Jul 19 11:09:29.311665 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Openstack-System-Scope: None Jul 19 11:09:29.311665 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: User-Agent: python-urllib3/1.26.9 Jul 19 11:09:29.311665 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Auth-Token: gAAAAABi1o_H80dr4M-lIIT6uzG8exhA4LKN984_YaeFvAcM9fuolWzoNXAcL7qhFNcYKUqYeIgZl9w0RVHbKPy5b2-j3djdpYY7wzh5yonBpziIeFPtCUxYkrQU736pgDwXX6Psqz45jhFgkLySjLfsmzUJa9qWn_nUPS3R9xp-POD7aFDj-oE Jul 19 11:09:29.311665 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Id: None Jul 19 11:09:29.311665 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Name: None Jul 19 11:09:29.311665 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Identity-Status: Confirmed Jul 19 11:09:29.311665 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Is-Admin-Project: True Jul 19 11:09:29.311665 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Openstack-Nova-Api-Version: 2.20 Jul 19 11:09:29.311665 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Id: default Jul 19 11:09:29.311665 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Name: Default Jul 19 11:09:29.311665 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Id: e1d59596ecae4e288733b90c577aa86b Jul 19 11:09:29.311665 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-36195510 Jul 19 11:09:29.311665 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Role: member,reader Jul 19 11:09:29.311665 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Roles: member,reader Jul 19 11:09:29.313031 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/e1d59596ecae4e288733b90c577aa86b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/e1d59596ecae4e288733b90c577aa86b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/e1d59596ecae4e288733b90c577aa86b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_e1d59596ecae4e288733b90c577aa86b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:09:29.313031 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant: tempest-AttachVolumeShelveTestJSON-36195510 Jul 19 11:09:29.313031 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Id: e1d59596ecae4e288733b90c577aa86b Jul 19 11:09:29.313031 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-36195510 Jul 19 11:09:29.313031 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User: tempest-AttachVolumeShelveTestJSON-36195510-project Jul 19 11:09:29.313031 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Id: default Jul 19 11:09:29.313031 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Name: Default Jul 19 11:09:29.313031 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Id: 246dc188d9a44b1cb38721e3dd8a863a Jul 19 11:09:29.313031 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Name: tempest-AttachVolumeShelveTestJSON-36195510-project Jul 19 11:09:29.313031 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: method: > Jul 19 11:09:29.313031 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: exception: Instance f52fbfae-577b-403d-a4d3-9a4b79ae2326 could not be found. Jul 19 11:09:29.313031 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: action_args: {'id': 'f52fbfae-577b-403d-a4d3-9a4b79ae2326'} Jul 19 11:09:29.313773 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:09:29.313773 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:09:29.313773 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:09:29.313773 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:09:29.313773 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:09:29.313773 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:09:29.313773 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:09:29.313773 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:09:29.313773 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:09:29.313773 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:09:29.313773 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f52fbfae-577b-403d-a4d3-9a4b79ae2326 could not be found. Jul 19 11:09:29.313773 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:09:29.313773 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:09:29.313773 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:09:29.313773 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:09:29.313773 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:09:29.313773 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:09:29.313773 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:09:29.313773 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:09:29.313773 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:09:29.315759 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:09:29.315759 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 19 11:09:29.315759 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 19 11:09:29.315759 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:09:29.315759 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:09:29.315759 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:09:29.315759 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:09:29.315759 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f52fbfae-577b-403d-a4d3-9a4b79ae2326 could not be found. Jul 19 11:09:29.315759 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:09:29.315759 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:09:29.315759 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:09:29.315759 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:09:29.315759 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:09:29.315759 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:09:29.315759 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:09:29.315759 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:09:29.315759 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f52fbfae-577b-403d-a4d3-9a4b79ae2326 could not be found. Jul 19 11:09:29.315759 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:09:29.317326 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-90cb6320-22cb-4b64-9aa4-2242061c156f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Returning 404 to user: Instance f52fbfae-577b-403d-a4d3-9a4b79ae2326 could not be found. {{(pid=117849) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:09:29.317326 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-90cb6320-22cb-4b64-9aa4-2242061c156f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 404 len: 111 microversion: 2.20 time: 0.033604 Jul 19 11:09:29.317326 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 379/782] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:09:29 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 19 11:09:29.322186 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-69df2b10-0128-4077-8f3f-05c468bd6ceb tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '.version_select at 0x7f04aa298430>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:29.323662 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-88cf4178-fa7a-4b45-97db-028320626962 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:29.327232 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:09:29.331769 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-88cf4178-fa7a-4b45-97db-028320626962 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1654 microversion: 2.1 time: 0.412862 Jul 19 11:09:29.332535 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 404/783] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:28 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1654 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:29.342668 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-69df2b10-0128-4077-8f3f-05c468bd6ceb tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-730599398" status: 204 len: 0 microversion: 2.20 time: 0.024075 Jul 19 11:09:29.343460 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 380/784] 10.222.0.33 () {60 vars in 1258 bytes} [Tue Jul 19 11:09:29 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-730599398 => generated 0 bytes in 26 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Jul 19 11:09:29.349853 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-48a9c581-4b24-426b-9636-ed00fd043817 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:09:29.357434 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-48a9c581-4b24-426b-9636-ed00fd043817 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:29.358127 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-48a9c581-4b24-426b-9636-ed00fd043817 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:29.383416 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-48a9c581-4b24-426b-9636-ed00fd043817 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Going to try to start instance {{(pid=117848) start /opt/stack/nova/nova/compute/api.py:2730}} Jul 19 11:09:29.432528 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-48a9c581-4b24-426b-9636-ed00fd043817 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "POST /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487/action" status: 202 len: 0 microversion: 2.1 time: 0.086222 Jul 19 11:09:29.433695 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 405/785] 10.222.0.33 () {60 vars in 1277 bytes} [Tue Jul 19 11:09:29 2022] POST /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487/action => generated 0 bytes in 88 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 19 11:09:29.443291 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-1701987e-c44a-41f1-80da-097af24cbadc tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:29.444169 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-1701987e-c44a-41f1-80da-097af24cbadc tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:29.454194 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1701987e-c44a-41f1-80da-097af24cbadc tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:29.455786 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1701987e-c44a-41f1-80da-097af24cbadc tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:29.456899 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1701987e-c44a-41f1-80da-097af24cbadc tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:29.457562 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1701987e-c44a-41f1-80da-097af24cbadc tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:29.639498 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-1701987e-c44a-41f1-80da-097af24cbadc tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:29.647288 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-1701987e-c44a-41f1-80da-097af24cbadc tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1663 microversion: 2.1 time: 0.209799 Jul 19 11:09:29.648101 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 381/786] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:29 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1663 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:30.136518 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-61aff054-ab85-4c05-86ee-1cadccfee427 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Action: 'create', calling method: .version_select at 0x7f04aa241550>, body: {"keypair": {"name": "tempest-keypair-1639386872", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBXIHrQQN2x5B+6GJGPhHSk206o9bnlwodKwjAFRSkMqxPrS22RpVtTciua1YHID5T7qDbKNY/SYsTA63Hr/ZbQuigR6nQgWjlYxEOJgdnVw58wnBsv4+FscAhHAJ/nxig=="}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:09:30.150756 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.quota [None req-61aff054-ab85-4c05-86ee-1cadccfee427 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Getting quotas for project e1d59596ecae4e288733b90c577aa86b. Resources: {'key_pairs'} {{(pid=117848) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:09:30.154592 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.quota [None req-61aff054-ab85-4c05-86ee-1cadccfee427 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Getting quotas for user 246dc188d9a44b1cb38721e3dd8a863a and project e1d59596ecae4e288733b90c577aa86b. Resources: {'key_pairs'} {{(pid=117848) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:09:30.175461 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-61aff054-ab85-4c05-86ee-1cadccfee427 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.043644 Jul 19 11:09:30.176050 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 406/787] 10.222.0.33 () {62 vars in 1199 bytes} [Tue Jul 19 11:09:30 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 45 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Jul 19 11:09:30.294840 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-2f554773-7faa-4e04-9da6-e439d208da23 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:30.295419 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-2f554773-7faa-4e04-9da6-e439d208da23 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:30.302145 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2f554773-7faa-4e04-9da6-e439d208da23 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:30.302594 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2f554773-7faa-4e04-9da6-e439d208da23 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:30.303350 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2f554773-7faa-4e04-9da6-e439d208da23 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:30.303782 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2f554773-7faa-4e04-9da6-e439d208da23 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:30.507638 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-2f554773-7faa-4e04-9da6-e439d208da23 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:30.517722 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-2f554773-7faa-4e04-9da6-e439d208da23 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1757 microversion: 2.1 time: 0.226728 Jul 19 11:09:30.518918 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 382/788] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:30 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1757 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:30.667327 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-192c7320-41f5-43aa-87fd-5460d3768b12 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:30.668130 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-192c7320-41f5-43aa-87fd-5460d3768b12 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:30.674851 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-192c7320-41f5-43aa-87fd-5460d3768b12 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:30.675297 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-192c7320-41f5-43aa-87fd-5460d3768b12 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:30.676066 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-192c7320-41f5-43aa-87fd-5460d3768b12 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:30.676501 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-192c7320-41f5-43aa-87fd-5460d3768b12 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:30.921411 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-192c7320-41f5-43aa-87fd-5460d3768b12 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:30.933229 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-192c7320-41f5-43aa-87fd-5460d3768b12 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1663 microversion: 2.1 time: 0.269944 Jul 19 11:09:30.933826 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 407/789] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:30 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1663 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:31.535488 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-ee2ca70f-2d5c-4989-ba16-e4bbd7ce73b8 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:31.536101 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-ee2ca70f-2d5c-4989-ba16-e4bbd7ce73b8 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:31.542134 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ee2ca70f-2d5c-4989-ba16-e4bbd7ce73b8 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:31.542590 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ee2ca70f-2d5c-4989-ba16-e4bbd7ce73b8 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:31.543360 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ee2ca70f-2d5c-4989-ba16-e4bbd7ce73b8 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:31.543803 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ee2ca70f-2d5c-4989-ba16-e4bbd7ce73b8 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:31.698176 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-ee2ca70f-2d5c-4989-ba16-e4bbd7ce73b8 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:31.709385 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-ee2ca70f-2d5c-4989-ba16-e4bbd7ce73b8 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1760 microversion: 2.1 time: 0.178260 Jul 19 11:09:31.710055 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 383/790] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:31 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1760 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:31.860020 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-944fd2ce-3585-4704-b2db-f2664eacbe10 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-985384684", "imageRef": "3085883e-2289-4314-aa47-9c41d4368b02", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "4e7c735a-1e35-464a-9187-a0d8149bbb40"}], "security_groups": [{"name": "tempest-securitygroup--1369458921"}], "key_name": "tempest-keypair-1639386872", "user_data": "}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:09:31.950209 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-ca2e571e-0841-4556-acb9-3c73e9c2aa19 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:31.951040 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-ca2e571e-0841-4556-acb9-3c73e9c2aa19 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:31.956976 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ca2e571e-0841-4556-acb9-3c73e9c2aa19 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:31.957433 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ca2e571e-0841-4556-acb9-3c73e9c2aa19 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:31.958204 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ca2e571e-0841-4556-acb9-3c73e9c2aa19 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:31.958660 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ca2e571e-0841-4556-acb9-3c73e9c2aa19 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:31.961474 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.network.neutron [None req-944fd2ce-3585-4704-b2db-f2664eacbe10 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] validate_networks() for [('4e7c735a-1e35-464a-9187-a0d8149bbb40', None, None, None, None, None)] {{(pid=117848) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 19 11:09:32.088380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-ca2e571e-0841-4556-acb9-3c73e9c2aa19 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:32.096221 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-ca2e571e-0841-4556-acb9-3c73e9c2aa19 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1663 microversion: 2.1 time: 0.150341 Jul 19 11:09:32.096790 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 384/791] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:31 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1663 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:09:32.336606 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-944fd2ce-3585-4704-b2db-f2664eacbe10 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Flavor limits 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 19 11:09:32.337070 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-944fd2ce-3585-4704-b2db-f2664eacbe10 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Image limits 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 19 11:09:32.337545 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-944fd2ce-3585-4704-b2db-f2664eacbe10 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Flavor pref 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 19 11:09:32.337953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-944fd2ce-3585-4704-b2db-f2664eacbe10 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Image pref 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 19 11:09:32.338364 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-944fd2ce-3585-4704-b2db-f2664eacbe10 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 19 11:09:32.347623 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-944fd2ce-3585-4704-b2db-f2664eacbe10 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:32.348065 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-944fd2ce-3585-4704-b2db-f2664eacbe10 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:32.348902 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-944fd2ce-3585-4704-b2db-f2664eacbe10 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:32.349335 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-944fd2ce-3585-4704-b2db-f2664eacbe10 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:32.366058 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.quota [None req-944fd2ce-3585-4704-b2db-f2664eacbe10 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Getting quotas for project e1d59596ecae4e288733b90c577aa86b. Resources: {'cores', 'instances', 'ram'} {{(pid=117848) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:09:32.369927 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.quota [None req-944fd2ce-3585-4704-b2db-f2664eacbe10 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Getting quotas for user 246dc188d9a44b1cb38721e3dd8a863a and project e1d59596ecae4e288733b90c577aa86b. Resources: {'cores', 'instances', 'ram'} {{(pid=117848) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:09:32.380817 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-944fd2ce-3585-4704-b2db-f2664eacbe10 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Going to run 1 instances... {{(pid=117848) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 19 11:09:32.383022 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-944fd2ce-3585-4704-b2db-f2664eacbe10 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='3085883e-2289-4314-aa47-9c41d4368b02',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117848) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 19 11:09:32.428822 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-944fd2ce-3585-4704-b2db-f2664eacbe10 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 0.573060 Jul 19 11:09:32.429475 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 408/792] 10.222.0.33 () {62 vars in 1187 bytes} [Tue Jul 19 11:09:31 2022] POST /compute/v2.1/servers => generated 402 bytes in 574 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Jul 19 11:09:32.439378 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-ee9592dc-6de4-45ce-80d8-57a716adeb2c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:32.440082 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-ee9592dc-6de4-45ce-80d8-57a716adeb2c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:32.494078 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-ee9592dc-6de4-45ce-80d8-57a716adeb2c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:32.501531 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-ee9592dc-6de4-45ce-80d8-57a716adeb2c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:32.503033 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-ee9592dc-6de4-45ce-80d8-57a716adeb2c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:32.504126 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-ee9592dc-6de4-45ce-80d8-57a716adeb2c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1281 microversion: 2.20 time: 0.068648 Jul 19 11:09:32.504681 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 385/793] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:09:32 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1281 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:09:33.114762 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-09b1a3ba-cfa1-4e33-aa75-c6716c89a322 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:33.115427 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-09b1a3ba-cfa1-4e33-aa75-c6716c89a322 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:33.118309 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_api:SELECT=95,nova_api:UPDATE=1,nova_api:INSERT=4 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:09:33.124633 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-09b1a3ba-cfa1-4e33-aa75-c6716c89a322 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:33.125118 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-09b1a3ba-cfa1-4e33-aa75-c6716c89a322 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:33.125899 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-09b1a3ba-cfa1-4e33-aa75-c6716c89a322 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:33.126326 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-09b1a3ba-cfa1-4e33-aa75-c6716c89a322 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:33.271221 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-09b1a3ba-cfa1-4e33-aa75-c6716c89a322 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:33.279085 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-09b1a3ba-cfa1-4e33-aa75-c6716c89a322 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1667 microversion: 2.1 time: 0.169034 Jul 19 11:09:33.279643 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 409/794] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:33 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1667 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:09:33.522201 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-bda53b2c-af80-4074-be7c-9ac35ee67828 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:33.522797 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-bda53b2c-af80-4074-be7c-9ac35ee67828 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:33.531116 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bda53b2c-af80-4074-be7c-9ac35ee67828 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:33.531556 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bda53b2c-af80-4074-be7c-9ac35ee67828 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:33.532313 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bda53b2c-af80-4074-be7c-9ac35ee67828 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:33.532766 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bda53b2c-af80-4074-be7c-9ac35ee67828 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:33.596680 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-bda53b2c-af80-4074-be7c-9ac35ee67828 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:33.605231 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-bda53b2c-af80-4074-be7c-9ac35ee67828 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:33.606729 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-bda53b2c-af80-4074-be7c-9ac35ee67828 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:33.607810 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-bda53b2c-af80-4074-be7c-9ac35ee67828 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1333 microversion: 2.20 time: 0.089841 Jul 19 11:09:33.608355 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 386/795] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:09:33 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1333 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:09:34.626657 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-7c3994f4-8bbb-46f6-a4a3-ec7c3c7a67a9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:34.627576 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-7c3994f4-8bbb-46f6-a4a3-ec7c3c7a67a9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:34.636145 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7c3994f4-8bbb-46f6-a4a3-ec7c3c7a67a9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:34.636757 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7c3994f4-8bbb-46f6-a4a3-ec7c3c7a67a9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:34.638356 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7c3994f4-8bbb-46f6-a4a3-ec7c3c7a67a9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:34.638851 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7c3994f4-8bbb-46f6-a4a3-ec7c3c7a67a9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:34.779862 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-7c3994f4-8bbb-46f6-a4a3-ec7c3c7a67a9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:34.787856 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-7c3994f4-8bbb-46f6-a4a3-ec7c3c7a67a9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:34.789485 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-7c3994f4-8bbb-46f6-a4a3-ec7c3c7a67a9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:34.790653 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-7c3994f4-8bbb-46f6-a4a3-ec7c3c7a67a9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1407 microversion: 2.20 time: 0.168309 Jul 19 11:09:34.791607 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 410/796] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:09:34 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1407 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:09:35.219141 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-913eb94f-01a5-4b86-9d19-e90ded392817 req-7289bd55-b2c1-4a98-aa6a-4ff8aaa91ed2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "36abf2ae-3b4c-4095-b9ba-2bd738e266d9", "tag": "049ef62d-5f12-4134-9010-2bc20edecd41"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:09:35.233824 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-913eb94f-01a5-4b86-9d19-e90ded392817 req-7289bd55-b2c1-4a98-aa6a-4ff8aaa91ed2 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:35.234294 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-913eb94f-01a5-4b86-9d19-e90ded392817 req-7289bd55-b2c1-4a98-aa6a-4ff8aaa91ed2 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:35.246412 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-913eb94f-01a5-4b86-9d19-e90ded392817 req-7289bd55-b2c1-4a98-aa6a-4ff8aaa91ed2 service nova] Creating event network-changed:049ef62d-5f12-4134-9010-2bc20edecd41 for instance 36abf2ae-3b4c-4095-b9ba-2bd738e266d9 on hv-cinder-maste Jul 19 11:09:35.252104 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-913eb94f-01a5-4b86-9d19-e90ded392817 req-7289bd55-b2c1-4a98-aa6a-4ff8aaa91ed2 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037153 Jul 19 11:09:35.263706 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 387/797] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:09:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:09:35.809164 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-07377910-d6ea-405d-ba33-0db53916648a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:35.809765 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-07377910-d6ea-405d-ba33-0db53916648a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:35.816276 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-07377910-d6ea-405d-ba33-0db53916648a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:35.816767 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-07377910-d6ea-405d-ba33-0db53916648a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:35.817529 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-07377910-d6ea-405d-ba33-0db53916648a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:35.817965 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-07377910-d6ea-405d-ba33-0db53916648a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:35.874680 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-94c61ebc-0d57-4116-8cb2-e985aebf9ab1 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:09:35.875210 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-94c61ebc-0d57-4116-8cb2-e985aebf9ab1 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:35.881474 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-94c61ebc-0d57-4116-8cb2-e985aebf9ab1 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:35.881925 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-94c61ebc-0d57-4116-8cb2-e985aebf9ab1 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:35.882702 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-94c61ebc-0d57-4116-8cb2-e985aebf9ab1 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:35.883145 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-94c61ebc-0d57-4116-8cb2-e985aebf9ab1 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:35.906106 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.objects.instance [None req-94c61ebc-0d57-4116-8cb2-e985aebf9ab1 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lazy-loading 'flavor' on Instance uuid 5b72a14b-bd18-4dfa-a68f-12a5424f471c {{(pid=117849) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 19 11:09:35.947282 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-07377910-d6ea-405d-ba33-0db53916648a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:35.955973 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-07377910-d6ea-405d-ba33-0db53916648a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:35.957560 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-07377910-d6ea-405d-ba33-0db53916648a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:35.958657 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-07377910-d6ea-405d-ba33-0db53916648a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1407 microversion: 2.20 time: 0.153412 Jul 19 11:09:35.959217 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 411/798] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:09:35 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1407 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:09:35.977740 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-94c61ebc-0d57-4116-8cb2-e985aebf9ab1 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "POST /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c/action" status: 202 len: 0 microversion: 2.1 time: 0.107601 Jul 19 11:09:35.978310 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 388/799] 10.222.0.33 () {60 vars in 1277 bytes} [Tue Jul 19 11:09:35 2022] POST /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c/action => generated 0 bytes in 109 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 19 11:09:35.986357 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-2fef4c8e-f68f-405b-a5d0-6267c7c35277 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:35.986958 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-2fef4c8e-f68f-405b-a5d0-6267c7c35277 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:35.992851 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2fef4c8e-f68f-405b-a5d0-6267c7c35277 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:35.993297 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2fef4c8e-f68f-405b-a5d0-6267c7c35277 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:35.994049 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2fef4c8e-f68f-405b-a5d0-6267c7c35277 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:35.994492 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2fef4c8e-f68f-405b-a5d0-6267c7c35277 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:36.128981 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-2fef4c8e-f68f-405b-a5d0-6267c7c35277 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:36.136508 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-2fef4c8e-f68f-405b-a5d0-6267c7c35277 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1567 microversion: 2.1 time: 0.153932 Jul 19 11:09:36.137699 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 412/800] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:35 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1567 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:36.979262 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-4d38a63a-02e5-4d5b-bbc0-dfd4729c1bb8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:36.979953 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-4d38a63a-02e5-4d5b-bbc0-dfd4729c1bb8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:36.986887 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4d38a63a-02e5-4d5b-bbc0-dfd4729c1bb8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:36.987719 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4d38a63a-02e5-4d5b-bbc0-dfd4729c1bb8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:36.988082 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4d38a63a-02e5-4d5b-bbc0-dfd4729c1bb8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:36.988617 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4d38a63a-02e5-4d5b-bbc0-dfd4729c1bb8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:37.126070 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-4d38a63a-02e5-4d5b-bbc0-dfd4729c1bb8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:37.133934 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-4d38a63a-02e5-4d5b-bbc0-dfd4729c1bb8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:37.135429 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-4d38a63a-02e5-4d5b-bbc0-dfd4729c1bb8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:37.136513 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-4d38a63a-02e5-4d5b-bbc0-dfd4729c1bb8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1407 microversion: 2.20 time: 0.162476 Jul 19 11:09:37.137125 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 389/801] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:09:36 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1407 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:09:37.154766 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-8b9b6e03-5ef7-4979-9dd6-54f313da498a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:37.155452 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-8b9b6e03-5ef7-4979-9dd6-54f313da498a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:37.162004 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8b9b6e03-5ef7-4979-9dd6-54f313da498a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:37.162538 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8b9b6e03-5ef7-4979-9dd6-54f313da498a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:37.163366 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8b9b6e03-5ef7-4979-9dd6-54f313da498a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:37.163871 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8b9b6e03-5ef7-4979-9dd6-54f313da498a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:37.293354 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-8b9b6e03-5ef7-4979-9dd6-54f313da498a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:37.300807 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-8b9b6e03-5ef7-4979-9dd6-54f313da498a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1567 microversion: 2.1 time: 0.150440 Jul 19 11:09:37.301368 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 413/802] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:37 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1567 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:38.154750 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-1f6c45f5-5494-4fb7-bf78-36d05f8a1f67 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:38.155342 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-1f6c45f5-5494-4fb7-bf78-36d05f8a1f67 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:38.161776 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1f6c45f5-5494-4fb7-bf78-36d05f8a1f67 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:38.162239 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1f6c45f5-5494-4fb7-bf78-36d05f8a1f67 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:38.162971 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1f6c45f5-5494-4fb7-bf78-36d05f8a1f67 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:38.163399 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1f6c45f5-5494-4fb7-bf78-36d05f8a1f67 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:38.289433 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-1f6c45f5-5494-4fb7-bf78-36d05f8a1f67 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:38.297349 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-1f6c45f5-5494-4fb7-bf78-36d05f8a1f67 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:38.298860 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-1f6c45f5-5494-4fb7-bf78-36d05f8a1f67 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:38.299942 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-1f6c45f5-5494-4fb7-bf78-36d05f8a1f67 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1407 microversion: 2.20 time: 0.149734 Jul 19 11:09:38.300500 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 390/803] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:09:38 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1407 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:09:38.317724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-e60ca6b9-0086-4c49-ac09-aac0fdf9daf3 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:38.318334 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-e60ca6b9-0086-4c49-ac09-aac0fdf9daf3 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:38.325410 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e60ca6b9-0086-4c49-ac09-aac0fdf9daf3 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:38.325858 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e60ca6b9-0086-4c49-ac09-aac0fdf9daf3 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:38.326620 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e60ca6b9-0086-4c49-ac09-aac0fdf9daf3 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:38.327057 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e60ca6b9-0086-4c49-ac09-aac0fdf9daf3 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:38.459854 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-e60ca6b9-0086-4c49-ac09-aac0fdf9daf3 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:38.467762 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-e60ca6b9-0086-4c49-ac09-aac0fdf9daf3 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1567 microversion: 2.1 time: 0.153649 Jul 19 11:09:38.468331 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 414/804] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:38 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1567 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:39.317639 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-5e6f3fe6-1a53-46a4-8fdb-4b6b6a510e74 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:39.318168 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-5e6f3fe6-1a53-46a4-8fdb-4b6b6a510e74 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:39.324541 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5e6f3fe6-1a53-46a4-8fdb-4b6b6a510e74 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:39.325020 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5e6f3fe6-1a53-46a4-8fdb-4b6b6a510e74 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:39.325769 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5e6f3fe6-1a53-46a4-8fdb-4b6b6a510e74 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:39.326234 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5e6f3fe6-1a53-46a4-8fdb-4b6b6a510e74 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:39.464863 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-5e6f3fe6-1a53-46a4-8fdb-4b6b6a510e74 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:39.473175 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-5e6f3fe6-1a53-46a4-8fdb-4b6b6a510e74 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:39.474714 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-5e6f3fe6-1a53-46a4-8fdb-4b6b6a510e74 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:39.475838 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-5e6f3fe6-1a53-46a4-8fdb-4b6b6a510e74 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1407 microversion: 2.20 time: 0.162138 Jul 19 11:09:39.476396 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 391/805] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:09:39 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1407 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:09:39.485138 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-974f259c-375a-49e3-ae54-44d045dc4204 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:39.485834 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-974f259c-375a-49e3-ae54-44d045dc4204 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:39.492447 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-974f259c-375a-49e3-ae54-44d045dc4204 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:39.492938 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-974f259c-375a-49e3-ae54-44d045dc4204 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:39.493703 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-974f259c-375a-49e3-ae54-44d045dc4204 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:39.494157 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-974f259c-375a-49e3-ae54-44d045dc4204 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:39.622667 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-974f259c-375a-49e3-ae54-44d045dc4204 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:39.630319 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-974f259c-375a-49e3-ae54-44d045dc4204 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1567 microversion: 2.1 time: 0.149683 Jul 19 11:09:39.630920 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 415/806] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:39 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1567 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:40.546772 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-bca15b8b-1d5b-42e0-a0a6-869de342e5c5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:40.547370 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-bca15b8b-1d5b-42e0-a0a6-869de342e5c5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:40.554530 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bca15b8b-1d5b-42e0-a0a6-869de342e5c5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:40.554972 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bca15b8b-1d5b-42e0-a0a6-869de342e5c5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:40.555729 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bca15b8b-1d5b-42e0-a0a6-869de342e5c5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:40.556155 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bca15b8b-1d5b-42e0-a0a6-869de342e5c5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:40.647125 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-ff2ff3ed-3fba-4274-9a2a-b34952da1f4e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:40.647724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-ff2ff3ed-3fba-4274-9a2a-b34952da1f4e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:40.654657 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ff2ff3ed-3fba-4274-9a2a-b34952da1f4e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:40.655110 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ff2ff3ed-3fba-4274-9a2a-b34952da1f4e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:40.655868 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ff2ff3ed-3fba-4274-9a2a-b34952da1f4e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:40.656299 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ff2ff3ed-3fba-4274-9a2a-b34952da1f4e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:40.687109 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-bca15b8b-1d5b-42e0-a0a6-869de342e5c5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:40.695458 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-bca15b8b-1d5b-42e0-a0a6-869de342e5c5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:40.697140 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-bca15b8b-1d5b-42e0-a0a6-869de342e5c5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:40.698322 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-bca15b8b-1d5b-42e0-a0a6-869de342e5c5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1407 microversion: 2.20 time: 0.208179 Jul 19 11:09:40.698927 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 392/807] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:09:40 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1407 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:09:40.788768 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-ff2ff3ed-3fba-4274-9a2a-b34952da1f4e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:40.796753 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-ff2ff3ed-3fba-4274-9a2a-b34952da1f4e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1567 microversion: 2.1 time: 0.153379 Jul 19 11:09:40.797313 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 416/808] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:40 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1567 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:09:41.715478 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-d7a179ae-09f1-4f65-9210-cd826862dc4e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:41.716073 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-d7a179ae-09f1-4f65-9210-cd826862dc4e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:41.722504 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d7a179ae-09f1-4f65-9210-cd826862dc4e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:41.722952 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d7a179ae-09f1-4f65-9210-cd826862dc4e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:41.723704 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d7a179ae-09f1-4f65-9210-cd826862dc4e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:41.724134 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d7a179ae-09f1-4f65-9210-cd826862dc4e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:41.814296 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-7561a455-b7ca-4550-abac-b625578ccd63 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:41.815000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-7561a455-b7ca-4550-abac-b625578ccd63 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:41.821639 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7561a455-b7ca-4550-abac-b625578ccd63 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:41.822229 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7561a455-b7ca-4550-abac-b625578ccd63 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:41.823093 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7561a455-b7ca-4550-abac-b625578ccd63 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:41.823597 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7561a455-b7ca-4550-abac-b625578ccd63 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:41.856060 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-d7a179ae-09f1-4f65-9210-cd826862dc4e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:41.865188 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-d7a179ae-09f1-4f65-9210-cd826862dc4e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:41.866718 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-d7a179ae-09f1-4f65-9210-cd826862dc4e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:41.867830 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-d7a179ae-09f1-4f65-9210-cd826862dc4e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1407 microversion: 2.20 time: 0.156260 Jul 19 11:09:41.868587 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 393/809] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:09:41 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1407 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:09:41.959857 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-7561a455-b7ca-4550-abac-b625578ccd63 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:41.967670 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-7561a455-b7ca-4550-abac-b625578ccd63 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1567 microversion: 2.1 time: 0.157212 Jul 19 11:09:41.968183 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 417/810] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:41 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1567 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:09:42.014822 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-43f77593-fe27-47e4-91b8-dc060179ecb8 req-fa999087-a77d-47af-af90-30094f14f015 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "36abf2ae-3b4c-4095-b9ba-2bd738e266d9", "name": "network-vif-plugged", "status": "completed", "tag": "049ef62d-5f12-4134-9010-2bc20edecd41"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:09:42.022751 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-43f77593-fe27-47e4-91b8-dc060179ecb8 req-fa999087-a77d-47af-af90-30094f14f015 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:42.023200 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-43f77593-fe27-47e4-91b8-dc060179ecb8 req-fa999087-a77d-47af-af90-30094f14f015 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:42.034383 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-43f77593-fe27-47e4-91b8-dc060179ecb8 req-fa999087-a77d-47af-af90-30094f14f015 service nova] Creating event network-vif-plugged:049ef62d-5f12-4134-9010-2bc20edecd41 for instance 36abf2ae-3b4c-4095-b9ba-2bd738e266d9 on hv-cinder-maste Jul 19 11:09:42.039275 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-43f77593-fe27-47e4-91b8-dc060179ecb8 req-fa999087-a77d-47af-af90-30094f14f015 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.028127 Jul 19 11:09:42.040088 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 394/811] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:09:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:09:42.886028 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-9fb4e784-2fa3-4c07-b9bc-e209f446e780 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:42.886568 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-9fb4e784-2fa3-4c07-b9bc-e209f446e780 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:42.888424 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell0:SELECT=2 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:09:42.889095 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell0:SELECT=10 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:09:42.895586 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9fb4e784-2fa3-4c07-b9bc-e209f446e780 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:42.896036 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9fb4e784-2fa3-4c07-b9bc-e209f446e780 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:42.896822 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9fb4e784-2fa3-4c07-b9bc-e209f446e780 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:42.897260 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9fb4e784-2fa3-4c07-b9bc-e209f446e780 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:42.982355 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell0:SELECT=2 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:09:42.987718 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-ed370ae7-0f00-4061-84ad-306b7395c2a0 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:42.988507 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-ed370ae7-0f00-4061-84ad-306b7395c2a0 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:42.990438 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_api:SELECT=83,nova_api:DELETE=1 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:09:42.996484 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ed370ae7-0f00-4061-84ad-306b7395c2a0 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:42.996962 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ed370ae7-0f00-4061-84ad-306b7395c2a0 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:42.997743 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ed370ae7-0f00-4061-84ad-306b7395c2a0 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:42.998184 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ed370ae7-0f00-4061-84ad-306b7395c2a0 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:43.025974 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-9fb4e784-2fa3-4c07-b9bc-e209f446e780 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:43.034452 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-9fb4e784-2fa3-4c07-b9bc-e209f446e780 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:43.035964 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-9fb4e784-2fa3-4c07-b9bc-e209f446e780 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:43.037087 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-9fb4e784-2fa3-4c07-b9bc-e209f446e780 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1407 microversion: 2.20 time: 0.155022 Jul 19 11:09:43.037649 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 418/812] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:09:42 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1407 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:09:43.131164 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-ed370ae7-0f00-4061-84ad-306b7395c2a0 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:43.134509 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:09:43.139831 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-ed370ae7-0f00-4061-84ad-306b7395c2a0 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1567 microversion: 2.1 time: 0.159073 Jul 19 11:09:43.140337 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 395/813] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:42 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1567 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:09:44.055358 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-8c81249c-0e07-491f-a0e0-ede3af0eb1b0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:44.055904 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-8c81249c-0e07-491f-a0e0-ede3af0eb1b0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:44.062914 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8c81249c-0e07-491f-a0e0-ede3af0eb1b0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:44.063380 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8c81249c-0e07-491f-a0e0-ede3af0eb1b0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:44.064128 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8c81249c-0e07-491f-a0e0-ede3af0eb1b0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:44.064557 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8c81249c-0e07-491f-a0e0-ede3af0eb1b0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:44.156582 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-e12200b2-1a89-4505-b9cc-6cc02b6e060a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:44.157240 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-e12200b2-1a89-4505-b9cc-6cc02b6e060a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:44.165467 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e12200b2-1a89-4505-b9cc-6cc02b6e060a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:44.165919 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e12200b2-1a89-4505-b9cc-6cc02b6e060a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:44.166691 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e12200b2-1a89-4505-b9cc-6cc02b6e060a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:44.167304 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e12200b2-1a89-4505-b9cc-6cc02b6e060a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:44.197228 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-8c81249c-0e07-491f-a0e0-ede3af0eb1b0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:44.205112 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-8c81249c-0e07-491f-a0e0-ede3af0eb1b0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:44.206610 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-8c81249c-0e07-491f-a0e0-ede3af0eb1b0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:44.207900 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-8c81249c-0e07-491f-a0e0-ede3af0eb1b0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1407 microversion: 2.20 time: 0.156483 Jul 19 11:09:44.208466 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 419/814] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:09:44 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1407 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:09:44.301858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-e12200b2-1a89-4505-b9cc-6cc02b6e060a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:44.310198 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-e12200b2-1a89-4505-b9cc-6cc02b6e060a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1567 microversion: 2.1 time: 0.157323 Jul 19 11:09:44.310705 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 396/815] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:44 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1567 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:09:44.740321 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-f721c9bf-0f25-4f87-93d7-49d3ac5cc93f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:44.740857 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-f721c9bf-0f25-4f87-93d7-49d3ac5cc93f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 53545a7c-c683-49d6-941d-4445f2b09379] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:44.747633 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f721c9bf-0f25-4f87-93d7-49d3ac5cc93f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:44.748079 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f721c9bf-0f25-4f87-93d7-49d3ac5cc93f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:44.748877 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f721c9bf-0f25-4f87-93d7-49d3ac5cc93f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:44.749566 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f721c9bf-0f25-4f87-93d7-49d3ac5cc93f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:44.920410 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-f721c9bf-0f25-4f87-93d7-49d3ac5cc93f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379/ips" status: 200 len: 150 microversion: 2.1 time: 0.233960 Jul 19 11:09:44.921000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 420/816] 10.222.0.33 () {58 vars in 1247 bytes} [Tue Jul 19 11:09:44 2022] GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379/ips => generated 150 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:09:45.225258 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-6fad86e5-8c19-486e-9a39-67309c3937bc tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:45.225861 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-6fad86e5-8c19-486e-9a39-67309c3937bc tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:45.232465 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6fad86e5-8c19-486e-9a39-67309c3937bc tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:45.232988 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6fad86e5-8c19-486e-9a39-67309c3937bc tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:45.233892 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6fad86e5-8c19-486e-9a39-67309c3937bc tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:45.234335 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6fad86e5-8c19-486e-9a39-67309c3937bc tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:45.327789 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-141ac4c8-a489-45c9-a1f2-d9ccfda79f96 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:45.328391 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-141ac4c8-a489-45c9-a1f2-d9ccfda79f96 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:45.335151 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-141ac4c8-a489-45c9-a1f2-d9ccfda79f96 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:45.335599 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-141ac4c8-a489-45c9-a1f2-d9ccfda79f96 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:45.336360 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-141ac4c8-a489-45c9-a1f2-d9ccfda79f96 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:45.336816 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-141ac4c8-a489-45c9-a1f2-d9ccfda79f96 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:45.362683 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-6fad86e5-8c19-486e-9a39-67309c3937bc tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:45.370683 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-6fad86e5-8c19-486e-9a39-67309c3937bc tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:45.372801 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-6fad86e5-8c19-486e-9a39-67309c3937bc tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:45.374191 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-6fad86e5-8c19-486e-9a39-67309c3937bc tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1598 microversion: 2.20 time: 0.152622 Jul 19 11:09:45.374753 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 397/817] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:09:45 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1598 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:09:45.391885 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-b68d0adb-08e1-4329-b79b-fa77816fa7ab tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:45.392373 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-b68d0adb-08e1-4329-b79b-fa77816fa7ab tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:45.398361 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b68d0adb-08e1-4329-b79b-fa77816fa7ab tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:45.398809 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b68d0adb-08e1-4329-b79b-fa77816fa7ab tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:45.399570 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b68d0adb-08e1-4329-b79b-fa77816fa7ab tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:45.400004 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b68d0adb-08e1-4329-b79b-fa77816fa7ab tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:45.467939 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-141ac4c8-a489-45c9-a1f2-d9ccfda79f96 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:45.475498 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-141ac4c8-a489-45c9-a1f2-d9ccfda79f96 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1567 microversion: 2.1 time: 0.152045 Jul 19 11:09:45.476056 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 421/818] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:45 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1567 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:09:45.482661 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-b68d0adb-08e1-4329-b79b-fa77816fa7ab tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.094581 Jul 19 11:09:45.483269 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 398/819] 10.222.0.33 () {60 vars in 1315 bytes} [Tue Jul 19 11:09:45 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9/os-interface => generated 284 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 19 11:09:46.235154 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-5483428f-933a-4617-b274-00a5e735023e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "39ebd92d-65eb-4b54-b004-23f982865be5"}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:09:46.236460 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-5483428f-933a-4617-b274-00a5e735023e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 53545a7c-c683-49d6-941d-4445f2b09379] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:46.242470 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5483428f-933a-4617-b274-00a5e735023e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:46.242924 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5483428f-933a-4617-b274-00a5e735023e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:46.243684 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5483428f-933a-4617-b274-00a5e735023e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:46.244119 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5483428f-933a-4617-b274-00a5e735023e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:46.260805 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:09:46.396972 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-aa2e36d2-c823-42a2-bb6f-b065cc3c723a req-59e843d7-e273-470c-8837-edf0f09e8df5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "36abf2ae-3b4c-4095-b9ba-2bd738e266d9", "tag": "049ef62d-5f12-4134-9010-2bc20edecd41"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:09:46.406178 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-aa2e36d2-c823-42a2-bb6f-b065cc3c723a req-59e843d7-e273-470c-8837-edf0f09e8df5 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:46.406621 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-aa2e36d2-c823-42a2-bb6f-b065cc3c723a req-59e843d7-e273-470c-8837-edf0f09e8df5 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:46.419281 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-aa2e36d2-c823-42a2-bb6f-b065cc3c723a req-59e843d7-e273-470c-8837-edf0f09e8df5 service nova] Creating event network-changed:049ef62d-5f12-4134-9010-2bc20edecd41 for instance 36abf2ae-3b4c-4095-b9ba-2bd738e266d9 on hv-cinder-maste Jul 19 11:09:46.424468 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-aa2e36d2-c823-42a2-bb6f-b065cc3c723a req-59e843d7-e273-470c-8837-edf0f09e8df5 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031999 Jul 19 11:09:46.425090 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 399/820] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:09:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:09:46.493576 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-b04011b1-f8bb-432e-8c94-ce29247e6a75 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:46.495277 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-b04011b1-f8bb-432e-8c94-ce29247e6a75 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:46.502802 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b04011b1-f8bb-432e-8c94-ce29247e6a75 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:46.503250 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b04011b1-f8bb-432e-8c94-ce29247e6a75 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:46.504073 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b04011b1-f8bb-432e-8c94-ce29247e6a75 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:46.505290 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b04011b1-f8bb-432e-8c94-ce29247e6a75 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:46.523305 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-5483428f-933a-4617-b274-00a5e735023e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "POST /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.293179 Jul 19 11:09:46.524904 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 422/821] 10.222.0.33 () {60 vars in 1322 bytes} [Tue Jul 19 11:09:46 2022] POST /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379/os-volume_attachments => generated 194 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:09:46.727305 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b04011b1-f8bb-432e-8c94-ce29247e6a75 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:46.735092 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-b04011b1-f8bb-432e-8c94-ce29247e6a75 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1567 microversion: 2.1 time: 0.246625 Jul 19 11:09:46.735664 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 400/822] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:46 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1567 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:09:47.753366 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-3c153314-0305-4077-b445-1e9a525d12c0 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:47.754155 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-3c153314-0305-4077-b445-1e9a525d12c0 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:47.760925 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3c153314-0305-4077-b445-1e9a525d12c0 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:47.761363 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3c153314-0305-4077-b445-1e9a525d12c0 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:47.762150 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3c153314-0305-4077-b445-1e9a525d12c0 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:47.762579 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3c153314-0305-4077-b445-1e9a525d12c0 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:47.895480 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-3c153314-0305-4077-b445-1e9a525d12c0 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:47.903095 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-3c153314-0305-4077-b445-1e9a525d12c0 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1567 microversion: 2.1 time: 0.154418 Jul 19 11:09:47.903656 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 423/823] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:47 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1567 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:09:48.920851 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-218f8557-312f-4171-956c-bf484f6d02c3 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:48.921412 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-218f8557-312f-4171-956c-bf484f6d02c3 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:48.927960 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-218f8557-312f-4171-956c-bf484f6d02c3 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:48.928407 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-218f8557-312f-4171-956c-bf484f6d02c3 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:48.929217 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-218f8557-312f-4171-956c-bf484f6d02c3 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:48.929651 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-218f8557-312f-4171-956c-bf484f6d02c3 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:49.057564 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-218f8557-312f-4171-956c-bf484f6d02c3 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:49.065018 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-218f8557-312f-4171-956c-bf484f6d02c3 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1567 microversion: 2.1 time: 0.148737 Jul 19 11:09:49.065577 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 401/824] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:48 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1567 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:50.081943 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-eefbc534-79f8-44a0-8e11-3147dc27eccb tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:50.082548 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-eefbc534-79f8-44a0-8e11-3147dc27eccb tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:50.088986 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-eefbc534-79f8-44a0-8e11-3147dc27eccb tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:50.089439 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-eefbc534-79f8-44a0-8e11-3147dc27eccb tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:50.090197 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-eefbc534-79f8-44a0-8e11-3147dc27eccb tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:50.090632 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-eefbc534-79f8-44a0-8e11-3147dc27eccb tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:50.218200 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-eefbc534-79f8-44a0-8e11-3147dc27eccb tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:50.225548 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-eefbc534-79f8-44a0-8e11-3147dc27eccb tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1567 microversion: 2.1 time: 0.147404 Jul 19 11:09:50.226107 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 424/825] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:50 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1567 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:51.242939 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-97c33fb1-9942-4e25-9706-348bfd9edd97 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:51.243546 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-97c33fb1-9942-4e25-9706-348bfd9edd97 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:51.250304 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-97c33fb1-9942-4e25-9706-348bfd9edd97 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:51.250757 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-97c33fb1-9942-4e25-9706-348bfd9edd97 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:51.251518 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-97c33fb1-9942-4e25-9706-348bfd9edd97 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:51.251950 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-97c33fb1-9942-4e25-9706-348bfd9edd97 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:51.381594 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-97c33fb1-9942-4e25-9706-348bfd9edd97 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:51.389152 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-97c33fb1-9942-4e25-9706-348bfd9edd97 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1508 microversion: 2.1 time: 0.150038 Jul 19 11:09:51.389712 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 402/826] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:51 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1508 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:51.405940 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-72be05d8-14bf-4528-a6c8-7065f09b1563 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:09:51.406516 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-72be05d8-14bf-4528-a6c8-7065f09b1563 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:51.413091 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-72be05d8-14bf-4528-a6c8-7065f09b1563 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:51.413545 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-72be05d8-14bf-4528-a6c8-7065f09b1563 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:51.414312 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-72be05d8-14bf-4528-a6c8-7065f09b1563 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:51.414766 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-72be05d8-14bf-4528-a6c8-7065f09b1563 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:51.489703 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-72be05d8-14bf-4528-a6c8-7065f09b1563 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "POST /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c/action" status: 202 len: 0 microversion: 2.1 time: 0.088118 Jul 19 11:09:51.490463 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 425/827] 10.222.0.33 () {60 vars in 1277 bytes} [Tue Jul 19 11:09:51 2022] POST /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c/action => generated 0 bytes in 89 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 19 11:09:51.498399 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-41e0ecac-2e09-41cc-8436-71bc2fee17bb tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:51.498936 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-41e0ecac-2e09-41cc-8436-71bc2fee17bb tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:51.505182 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-41e0ecac-2e09-41cc-8436-71bc2fee17bb tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:51.505642 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-41e0ecac-2e09-41cc-8436-71bc2fee17bb tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:51.506440 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-41e0ecac-2e09-41cc-8436-71bc2fee17bb tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:51.506882 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-41e0ecac-2e09-41cc-8436-71bc2fee17bb tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:51.678276 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-41e0ecac-2e09-41cc-8436-71bc2fee17bb tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:51.685925 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-41e0ecac-2e09-41cc-8436-71bc2fee17bb tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1516 microversion: 2.1 time: 0.191451 Jul 19 11:09:51.686490 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 403/828] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:51 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1516 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:09:52.703403 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-0216480c-fc3b-4177-8939-1e4e9e23ea1a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:52.703946 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-0216480c-fc3b-4177-8939-1e4e9e23ea1a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:52.709905 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0216480c-fc3b-4177-8939-1e4e9e23ea1a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:52.710347 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0216480c-fc3b-4177-8939-1e4e9e23ea1a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:52.711097 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0216480c-fc3b-4177-8939-1e4e9e23ea1a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:52.711561 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0216480c-fc3b-4177-8939-1e4e9e23ea1a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:52.838322 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-0216480c-fc3b-4177-8939-1e4e9e23ea1a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:52.846269 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-0216480c-fc3b-4177-8939-1e4e9e23ea1a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1518 microversion: 2.1 time: 0.147238 Jul 19 11:09:52.846823 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 426/829] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:52 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1518 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:53.863409 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-e0c8e99d-d2d4-4688-97b5-d0583561a206 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:53.864023 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-e0c8e99d-d2d4-4688-97b5-d0583561a206 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:53.870114 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e0c8e99d-d2d4-4688-97b5-d0583561a206 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:53.870578 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e0c8e99d-d2d4-4688-97b5-d0583561a206 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:53.871979 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e0c8e99d-d2d4-4688-97b5-d0583561a206 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:53.872424 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e0c8e99d-d2d4-4688-97b5-d0583561a206 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:54.018593 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-e0c8e99d-d2d4-4688-97b5-d0583561a206 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:54.026765 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-e0c8e99d-d2d4-4688-97b5-d0583561a206 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1518 microversion: 2.1 time: 0.167275 Jul 19 11:09:54.027610 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 404/830] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:53 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1518 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:54.047422 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-683ecae0-971f-4b47-b412-548c53ec98d4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "39ebd92d-65eb-4b54-b004-23f982865be5"}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:09:54.048532 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-683ecae0-971f-4b47-b412-548c53ec98d4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 53545a7c-c683-49d6-941d-4445f2b09379] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:54.055227 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-683ecae0-971f-4b47-b412-548c53ec98d4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:54.055551 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-683ecae0-971f-4b47-b412-548c53ec98d4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:54.056231 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-683ecae0-971f-4b47-b412-548c53ec98d4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:54.056659 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-683ecae0-971f-4b47-b412-548c53ec98d4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:54.090738 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-b05f1015-e64c-4a6d-b125-e908ee5d4323 req-755589b2-80b5-497a-96aa-6d830525f2db service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5b72a14b-bd18-4dfa-a68f-12a5424f471c", "name": "network-vif-unplugged", "status": "completed", "tag": "da843419-4401-4a26-82c2-7930b51aa1b8"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:09:54.098941 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-b05f1015-e64c-4a6d-b125-e908ee5d4323 req-755589b2-80b5-497a-96aa-6d830525f2db service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:54.099390 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-b05f1015-e64c-4a6d-b125-e908ee5d4323 req-755589b2-80b5-497a-96aa-6d830525f2db service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:54.113858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.compute.server_external_events [req-b05f1015-e64c-4a6d-b125-e908ee5d4323 req-755589b2-80b5-497a-96aa-6d830525f2db service nova] Unable to find a host for instance 5b72a14b-bd18-4dfa-a68f-12a5424f471c. Dropping event network-vif-unplugged {{(pid=117849) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:143}} Jul 19 11:09:54.114833 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-b05f1015-e64c-4a6d-b125-e908ee5d4323 req-755589b2-80b5-497a-96aa-6d830525f2db service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.028350 Jul 19 11:09:54.116024 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 405/831] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:09:54 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 30 msecs (HTTP/1.1 207) 9 headers in 367 bytes (1 switches on core 0) Jul 19 11:09:54.138973 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.wsgi [None req-683ecae0-971f-4b47-b412-548c53ec98d4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] HTTP exception thrown: Invalid volume: volume 39ebd92d-65eb-4b54-b004-23f982865be5 is already attached to instances: 53545a7c-c683-49d6-941d-4445f2b09379 Jul 19 11:09:54.141844 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-683ecae0-971f-4b47-b412-548c53ec98d4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Request method failure captured: Jul 19 11:09:54.141844 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: request: POST /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379/os-volume_attachments HTTP/1.1 Jul 19 11:09:54.141844 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept: application/json Jul 19 11:09:54.141844 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept-Encoding: identity Jul 19 11:09:54.141844 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Connection: close Jul 19 11:09:54.141844 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Length: 74 Jul 19 11:09:54.141844 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Type: application/json Jul 19 11:09:54.141844 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Host: 10.222.0.33 Jul 19 11:09:54.141844 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Openstack-System-Scope: None Jul 19 11:09:54.141844 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: User-Agent: python-urllib3/1.26.9 Jul 19 11:09:54.141844 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Auth-Token: gAAAAABi1o_HYpL7r_umyWTTKsnaQkfxgRQPrCVzXYV3YxPvpHKAUsKkXxPNkPyHBhEDbahQjtwakXlDpqARj9YKw3tFYmG0gCU4SGJZDCiqyWHloRDABrhK0mtm6JR4S49vBX-Zeg1oz_jYGWaYn6kU0GfIXikwtQO5JQ3VMjNcjqJ15uYlE9Q Jul 19 11:09:54.141844 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Id: None Jul 19 11:09:54.141844 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Name: None Jul 19 11:09:54.141844 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Identity-Status: Confirmed Jul 19 11:09:54.141844 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Is-Admin-Project: True Jul 19 11:09:54.141844 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Id: default Jul 19 11:09:54.141844 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Name: Default Jul 19 11:09:54.141844 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Id: ef534f1d06ea415d98857ec7f9e5fb97 Jul 19 11:09:54.141844 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Name: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:09:54.141844 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Role: member,reader Jul 19 11:09:54.141844 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Roles: member,reader Jul 19 11:09:54.143238 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:09:54.143238 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:09:54.143238 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Id: ef534f1d06ea415d98857ec7f9e5fb97 Jul 19 11:09:54.143238 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:09:54.143238 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User: tempest-AttachVolumeNegativeTest-434278602-project Jul 19 11:09:54.143238 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Id: default Jul 19 11:09:54.143238 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Name: Default Jul 19 11:09:54.143238 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Id: b8f2a78f3a3e4b8e8937e3bc39524690 Jul 19 11:09:54.143238 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Name: tempest-AttachVolumeNegativeTest-434278602-project Jul 19 11:09:54.143238 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Jul 19 11:09:54.143238 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: {"volumeAttachment": {"volumeId": "39ebd92d-65eb-4b54-b004-23f982865be5"}} Jul 19 11:09:54.143238 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: method: > Jul 19 11:09:54.143965 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: exception: Invalid volume: volume 39ebd92d-65eb-4b54-b004-23f982865be5 is already attached to instances: 53545a7c-c683-49d6-941d-4445f2b09379 Jul 19 11:09:54.143965 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: action_args: {'server_id': '53545a7c-c683-49d6-941d-4445f2b09379', 'body': {'volumeAttachment': {'volumeId': '39ebd92d-65eb-4b54-b004-23f982865be5'}}} Jul 19 11:09:54.143965 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:09:54.143965 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:09:54.143965 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Jul 19 11:09:54.143965 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Jul 19 11:09:54.143965 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 19 11:09:54.143965 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 19 11:09:54.143965 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 19 11:09:54.143965 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 19 11:09:54.143965 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4985, in attach_volume Jul 19 11:09:54.143965 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Jul 19 11:09:54.143965 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4825, in _check_volume_already_attached Jul 19 11:09:54.143965 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Jul 19 11:09:54.143965 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 39ebd92d-65eb-4b54-b004-23f982865be5 is already attached to instances: 53545a7c-c683-49d6-941d-4445f2b09379 Jul 19 11:09:54.143965 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:09:54.143965 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:09:54.143965 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:09:54.143965 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:09:54.145124 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:09:54.145124 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:09:54.145124 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:09:54.145124 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:09:54.145124 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:09:54.145124 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:09:54.145124 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 19 11:09:54.145124 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 19 11:09:54.145124 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 19 11:09:54.145124 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 19 11:09:54.145124 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 19 11:09:54.145124 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 19 11:09:54.145124 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Jul 19 11:09:54.145124 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 19 11:09:54.145124 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 39ebd92d-65eb-4b54-b004-23f982865be5 is already attached to instances: 53545a7c-c683-49d6-941d-4445f2b09379 Jul 19 11:09:54.145124 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:09:54.145124 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:09:54.145124 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:09:54.145124 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:09:54.145124 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:09:54.146360 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:09:54.146360 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:09:54.146360 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:09:54.146360 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 39ebd92d-65eb-4b54-b004-23f982865be5 is already attached to instances: 53545a7c-c683-49d6-941d-4445f2b09379 Jul 19 11:09:54.146360 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:09:54.146360 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-683ecae0-971f-4b47-b412-548c53ec98d4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Returning 400 to user: Invalid volume: volume 39ebd92d-65eb-4b54-b004-23f982865be5 is already attached to instances: 53545a7c-c683-49d6-941d-4445f2b09379 {{(pid=117848) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:09:54.146360 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-683ecae0-971f-4b47-b412-548c53ec98d4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "POST /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.100720 Jul 19 11:09:54.146360 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 427/832] 10.222.0.33 () {60 vars in 1322 bytes} [Tue Jul 19 11:09:54 2022] POST /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379/os-volume_attachments => generated 174 bytes in 102 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 19 11:09:54.204057 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-cf6b225b-fcdc-4e49-8670-6721965a81ee tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:54.204542 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-cf6b225b-fcdc-4e49-8670-6721965a81ee tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 53545a7c-c683-49d6-941d-4445f2b09379] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:54.210580 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cf6b225b-fcdc-4e49-8670-6721965a81ee tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:54.211024 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cf6b225b-fcdc-4e49-8670-6721965a81ee tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:54.211800 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cf6b225b-fcdc-4e49-8670-6721965a81ee tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:54.212229 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cf6b225b-fcdc-4e49-8670-6721965a81ee tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:54.291579 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.objects.instance [None req-cf6b225b-fcdc-4e49-8670-6721965a81ee tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lazy-loading 'services' on Instance uuid 53545a7c-c683-49d6-941d-4445f2b09379 {{(pid=117849) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 19 11:09:54.362127 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-cf6b225b-fcdc-4e49-8670-6721965a81ee tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "DELETE /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379/os-volume_attachments/39ebd92d-65eb-4b54-b004-23f982865be5" status: 202 len: 0 microversion: 2.1 time: 0.162422 Jul 19 11:09:54.362710 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 406/833] 10.222.0.33 () {58 vars in 1415 bytes} [Tue Jul 19 11:09:54 2022] DELETE /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379/os-volume_attachments/39ebd92d-65eb-4b54-b004-23f982865be5 => generated 0 bytes in 164 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 19 11:09:54.371624 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-d654fab4-26d2-476a-8ac4-a6de3df76a13 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:54.372551 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-d654fab4-26d2-476a-8ac4-a6de3df76a13 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 53545a7c-c683-49d6-941d-4445f2b09379] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:54.379304 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d654fab4-26d2-476a-8ac4-a6de3df76a13 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:54.379873 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d654fab4-26d2-476a-8ac4-a6de3df76a13 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:54.381127 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d654fab4-26d2-476a-8ac4-a6de3df76a13 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:54.381577 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d654fab4-26d2-476a-8ac4-a6de3df76a13 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:54.421108 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-d654fab4-26d2-476a-8ac4-a6de3df76a13 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053682 Jul 19 11:09:54.422053 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 428/834] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:09:54 2022] GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:09:55.045700 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-f15b3b93-6ffb-4778-83e3-5c9f9d7dd952 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:55.046382 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-f15b3b93-6ffb-4778-83e3-5c9f9d7dd952 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:55.053544 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f15b3b93-6ffb-4778-83e3-5c9f9d7dd952 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:55.053987 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f15b3b93-6ffb-4778-83e3-5c9f9d7dd952 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:55.054741 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f15b3b93-6ffb-4778-83e3-5c9f9d7dd952 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:55.055171 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f15b3b93-6ffb-4778-83e3-5c9f9d7dd952 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:55.194249 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-f15b3b93-6ffb-4778-83e3-5c9f9d7dd952 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:55.201828 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-f15b3b93-6ffb-4778-83e3-5c9f9d7dd952 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1518 microversion: 2.1 time: 0.161536 Jul 19 11:09:55.202390 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 407/835] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:55 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1518 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:55.432829 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-1d33f7c9-7aa1-4720-a384-4dcd8c754b77 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:55.433748 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-1d33f7c9-7aa1-4720-a384-4dcd8c754b77 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 53545a7c-c683-49d6-941d-4445f2b09379] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:55.440169 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1d33f7c9-7aa1-4720-a384-4dcd8c754b77 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:55.440616 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1d33f7c9-7aa1-4720-a384-4dcd8c754b77 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:55.441472 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1d33f7c9-7aa1-4720-a384-4dcd8c754b77 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:55.441862 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1d33f7c9-7aa1-4720-a384-4dcd8c754b77 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:55.471219 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-1d33f7c9-7aa1-4720-a384-4dcd8c754b77 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042356 Jul 19 11:09:55.471781 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 429/836] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:09:55 2022] GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:09:56.220284 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-53ee9830-5e28-4b0b-bbf2-e91bf1ed444e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:56.220920 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-53ee9830-5e28-4b0b-bbf2-e91bf1ed444e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:56.227269 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-53ee9830-5e28-4b0b-bbf2-e91bf1ed444e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:56.227718 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-53ee9830-5e28-4b0b-bbf2-e91bf1ed444e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:56.228475 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-53ee9830-5e28-4b0b-bbf2-e91bf1ed444e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:56.228940 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-53ee9830-5e28-4b0b-bbf2-e91bf1ed444e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:56.397778 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-53ee9830-5e28-4b0b-bbf2-e91bf1ed444e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:56.405192 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-53ee9830-5e28-4b0b-bbf2-e91bf1ed444e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1574 microversion: 2.1 time: 0.188760 Jul 19 11:09:56.405747 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 408/837] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:56 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1574 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:56.483618 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-d6ef0baa-7c3c-4c8d-8258-3734ac70f7c6 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:56.484536 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-d6ef0baa-7c3c-4c8d-8258-3734ac70f7c6 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 53545a7c-c683-49d6-941d-4445f2b09379] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:56.491264 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d6ef0baa-7c3c-4c8d-8258-3734ac70f7c6 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:56.491718 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d6ef0baa-7c3c-4c8d-8258-3734ac70f7c6 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:56.492496 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d6ef0baa-7c3c-4c8d-8258-3734ac70f7c6 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:56.492962 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d6ef0baa-7c3c-4c8d-8258-3734ac70f7c6 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:56.522903 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-d6ef0baa-7c3c-4c8d-8258-3734ac70f7c6 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.043259 Jul 19 11:09:56.523456 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 430/838] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:09:56 2022] GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379/os-volume_attachments => generated 25 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 19 11:09:56.579327 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-0193db94-dbb8-4449-8a30-4ae742b316d4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:56.579855 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-0193db94-dbb8-4449-8a30-4ae742b316d4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 53545a7c-c683-49d6-941d-4445f2b09379] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:56.585887 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0193db94-dbb8-4449-8a30-4ae742b316d4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:56.586288 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0193db94-dbb8-4449-8a30-4ae742b316d4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:56.587075 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0193db94-dbb8-4449-8a30-4ae742b316d4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:56.587510 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0193db94-dbb8-4449-8a30-4ae742b316d4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:56.611131 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-0193db94-dbb8-4449-8a30-4ae742b316d4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 53545a7c-c683-49d6-941d-4445f2b09379] Going to try to terminate instance {{(pid=117849) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 19 11:09:56.684082 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-0193db94-dbb8-4449-8a30-4ae742b316d4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "DELETE /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379" status: 204 len: 0 microversion: 2.1 time: 0.107797 Jul 19 11:09:56.684652 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 409/839] 10.222.0.33 () {58 vars in 1238 bytes} [Tue Jul 19 11:09:56 2022] DELETE /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 19 11:09:56.692887 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-f2f777e4-9302-4e19-b6c3-ae873393003d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:56.693497 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-f2f777e4-9302-4e19-b6c3-ae873393003d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 53545a7c-c683-49d6-941d-4445f2b09379] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:56.700027 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f2f777e4-9302-4e19-b6c3-ae873393003d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:56.700476 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f2f777e4-9302-4e19-b6c3-ae873393003d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:56.701265 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f2f777e4-9302-4e19-b6c3-ae873393003d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:56.701699 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f2f777e4-9302-4e19-b6c3-ae873393003d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:56.835406 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-f2f777e4-9302-4e19-b6c3-ae873393003d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:56.842964 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-f2f777e4-9302-4e19-b6c3-ae873393003d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379" status: 200 len: 1683 microversion: 2.1 time: 0.153876 Jul 19 11:09:56.843448 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 431/840] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:56 2022] GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379 => generated 1683 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:57.423365 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-d57815d1-0e42-4be1-8d8d-23f4f8f4b27f tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:57.424059 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-d57815d1-0e42-4be1-8d8d-23f4f8f4b27f tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:57.430549 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d57815d1-0e42-4be1-8d8d-23f4f8f4b27f tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:57.430996 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d57815d1-0e42-4be1-8d8d-23f4f8f4b27f tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:57.432530 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d57815d1-0e42-4be1-8d8d-23f4f8f4b27f tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:57.432994 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d57815d1-0e42-4be1-8d8d-23f4f8f4b27f tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:57.564981 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-d57815d1-0e42-4be1-8d8d-23f4f8f4b27f tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:57.572403 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-d57815d1-0e42-4be1-8d8d-23f4f8f4b27f tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1574 microversion: 2.1 time: 0.153848 Jul 19 11:09:57.572996 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 410/841] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:57 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1574 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:57.859878 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-eb9d4349-172d-4050-8941-91b7ad97172a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:57.860487 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-eb9d4349-172d-4050-8941-91b7ad97172a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 53545a7c-c683-49d6-941d-4445f2b09379] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:57.866984 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-eb9d4349-172d-4050-8941-91b7ad97172a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:57.867436 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-eb9d4349-172d-4050-8941-91b7ad97172a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:57.868197 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-eb9d4349-172d-4050-8941-91b7ad97172a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:57.868632 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-eb9d4349-172d-4050-8941-91b7ad97172a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:57.995881 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-eb9d4349-172d-4050-8941-91b7ad97172a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:58.004105 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-eb9d4349-172d-4050-8941-91b7ad97172a tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379" status: 200 len: 1683 microversion: 2.1 time: 0.148171 Jul 19 11:09:58.004666 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 432/842] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:57 2022] GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379 => generated 1683 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:58.589380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-3bb6c1dc-3baf-4836-9d97-0994e51c8870 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:58.590487 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-3bb6c1dc-3baf-4836-9d97-0994e51c8870 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:58.596917 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3bb6c1dc-3baf-4836-9d97-0994e51c8870 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:58.597369 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3bb6c1dc-3baf-4836-9d97-0994e51c8870 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:58.598126 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3bb6c1dc-3baf-4836-9d97-0994e51c8870 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:58.598560 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3bb6c1dc-3baf-4836-9d97-0994e51c8870 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:58.725861 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-3bb6c1dc-3baf-4836-9d97-0994e51c8870 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:58.729379 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:09:58.735007 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-3bb6c1dc-3baf-4836-9d97-0994e51c8870 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1574 microversion: 2.1 time: 0.149492 Jul 19 11:09:58.735569 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 411/843] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:58 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1574 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:59.021464 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-e53878d0-8314-407d-89ed-38ff1f57e516 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:59.022293 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-e53878d0-8314-407d-89ed-38ff1f57e516 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 53545a7c-c683-49d6-941d-4445f2b09379] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:59.029315 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e53878d0-8314-407d-89ed-38ff1f57e516 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:59.029762 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e53878d0-8314-407d-89ed-38ff1f57e516 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:59.030609 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e53878d0-8314-407d-89ed-38ff1f57e516 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:59.031137 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e53878d0-8314-407d-89ed-38ff1f57e516 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:59.169349 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-e53878d0-8314-407d-89ed-38ff1f57e516 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:59.177037 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-e53878d0-8314-407d-89ed-38ff1f57e516 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379" status: 200 len: 1683 microversion: 2.1 time: 0.159389 Jul 19 11:09:59.177602 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 433/844] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:59 2022] GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379 => generated 1683 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:09:59.649849 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-25f4c868-6d6c-449b-a3ac-76b653ff7114 req-6aafae4a-1127-4ef4-b3cb-491b01422067 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "53545a7c-c683-49d6-941d-4445f2b09379", "name": "network-vif-deleted", "tag": "bd4b190f-3901-45e6-89a0-d4cda5892170"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:09:59.663533 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-25f4c868-6d6c-449b-a3ac-76b653ff7114 req-6aafae4a-1127-4ef4-b3cb-491b01422067 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:59.664213 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-25f4c868-6d6c-449b-a3ac-76b653ff7114 req-6aafae4a-1127-4ef4-b3cb-491b01422067 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:59.677379 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-25f4c868-6d6c-449b-a3ac-76b653ff7114 req-6aafae4a-1127-4ef4-b3cb-491b01422067 service nova] Creating event network-vif-deleted:bd4b190f-3901-45e6-89a0-d4cda5892170 for instance 53545a7c-c683-49d6-941d-4445f2b09379 on hv-cinder-maste Jul 19 11:09:59.686738 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-25f4c868-6d6c-449b-a3ac-76b653ff7114 req-6aafae4a-1127-4ef4-b3cb-491b01422067 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.104490 Jul 19 11:09:59.687408 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 412/845] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:09:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:09:59.759986 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-e8af03f4-6516-4726-9a2d-c3044105bf22 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:09:59.761290 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-e8af03f4-6516-4726-9a2d-c3044105bf22 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:09:59.771731 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e8af03f4-6516-4726-9a2d-c3044105bf22 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:59.772454 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e8af03f4-6516-4726-9a2d-c3044105bf22 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:59.773555 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e8af03f4-6516-4726-9a2d-c3044105bf22 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:09:59.774151 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e8af03f4-6516-4726-9a2d-c3044105bf22 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:09:59.916241 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-e8af03f4-6516-4726-9a2d-c3044105bf22 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:09:59.923781 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-e8af03f4-6516-4726-9a2d-c3044105bf22 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1574 microversion: 2.1 time: 0.175594 Jul 19 11:09:59.924347 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 434/846] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:09:59 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1574 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:00.194777 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-eb9b676c-90e0-4600-a045-424f081a0a8c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:00.195418 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-eb9b676c-90e0-4600-a045-424f081a0a8c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 53545a7c-c683-49d6-941d-4445f2b09379] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:00.201817 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-eb9b676c-90e0-4600-a045-424f081a0a8c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:00.202272 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-eb9b676c-90e0-4600-a045-424f081a0a8c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:00.203072 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-eb9b676c-90e0-4600-a045-424f081a0a8c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:00.203508 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-eb9b676c-90e0-4600-a045-424f081a0a8c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:00.244242 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-eb9b676c-90e0-4600-a045-424f081a0a8c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:00.244686 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-eb9b676c-90e0-4600-a045-424f081a0a8c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:00.245159 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.objects.instance [None req-eb9b676c-90e0-4600-a045-424f081a0a8c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lazy-loading 'fault' on Instance uuid 53545a7c-c683-49d6-941d-4445f2b09379 {{(pid=117849) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 19 11:10:00.290545 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-eb9b676c-90e0-4600-a045-424f081a0a8c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:00.297881 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-eb9b676c-90e0-4600-a045-424f081a0a8c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379" status: 200 len: 1332 microversion: 2.1 time: 0.107516 Jul 19 11:10:00.298661 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 413/847] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:00 2022] GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379 => generated 1332 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:00.939733 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-7bc19dac-920e-436d-b8c9-725f6d7de520 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:00.940338 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-7bc19dac-920e-436d-b8c9-725f6d7de520 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:00.946784 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7bc19dac-920e-436d-b8c9-725f6d7de520 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:00.947230 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7bc19dac-920e-436d-b8c9-725f6d7de520 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:00.948015 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7bc19dac-920e-436d-b8c9-725f6d7de520 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:00.948443 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7bc19dac-920e-436d-b8c9-725f6d7de520 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:01.074241 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-7bc19dac-920e-436d-b8c9-725f6d7de520 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:01.082028 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-7bc19dac-920e-436d-b8c9-725f6d7de520 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1574 microversion: 2.1 time: 0.146126 Jul 19 11:10:01.082578 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 435/848] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:00 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1574 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:01.314351 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-d80439ee-8e70-422a-89ff-743bdab37f17 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:01.314871 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-d80439ee-8e70-422a-89ff-743bdab37f17 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 53545a7c-c683-49d6-941d-4445f2b09379] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:01.321277 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d80439ee-8e70-422a-89ff-743bdab37f17 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:01.321718 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d80439ee-8e70-422a-89ff-743bdab37f17 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:01.322468 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d80439ee-8e70-422a-89ff-743bdab37f17 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:01.322892 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d80439ee-8e70-422a-89ff-743bdab37f17 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:01.336396 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.wsgi [None req-d80439ee-8e70-422a-89ff-743bdab37f17 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] HTTP exception thrown: Instance 53545a7c-c683-49d6-941d-4445f2b09379 could not be found. Jul 19 11:10:01.338008 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-d80439ee-8e70-422a-89ff-743bdab37f17 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Request method failure captured: Jul 19 11:10:01.338008 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: request: GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379 HTTP/1.1 Jul 19 11:10:01.338008 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept: application/json Jul 19 11:10:01.338008 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept-Encoding: identity Jul 19 11:10:01.338008 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Connection: close Jul 19 11:10:01.338008 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Length: 0 Jul 19 11:10:01.338008 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Type: application/json Jul 19 11:10:01.338008 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Host: 10.222.0.33 Jul 19 11:10:01.338008 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Openstack-System-Scope: None Jul 19 11:10:01.338008 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: User-Agent: python-urllib3/1.26.9 Jul 19 11:10:01.338008 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Auth-Token: gAAAAABi1o_HYpL7r_umyWTTKsnaQkfxgRQPrCVzXYV3YxPvpHKAUsKkXxPNkPyHBhEDbahQjtwakXlDpqARj9YKw3tFYmG0gCU4SGJZDCiqyWHloRDABrhK0mtm6JR4S49vBX-Zeg1oz_jYGWaYn6kU0GfIXikwtQO5JQ3VMjNcjqJ15uYlE9Q Jul 19 11:10:01.338008 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Id: None Jul 19 11:10:01.338008 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Name: None Jul 19 11:10:01.338008 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Identity-Status: Confirmed Jul 19 11:10:01.338008 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Is-Admin-Project: True Jul 19 11:10:01.338008 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Id: default Jul 19 11:10:01.338008 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Name: Default Jul 19 11:10:01.338008 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Id: ef534f1d06ea415d98857ec7f9e5fb97 Jul 19 11:10:01.338008 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Name: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:10:01.338008 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Role: member,reader Jul 19 11:10:01.338008 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Roles: member,reader Jul 19 11:10:01.339480 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:10:01.339480 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:10:01.339480 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Id: ef534f1d06ea415d98857ec7f9e5fb97 Jul 19 11:10:01.339480 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:10:01.339480 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User: tempest-AttachVolumeNegativeTest-434278602-project Jul 19 11:10:01.339480 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Id: default Jul 19 11:10:01.339480 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Name: Default Jul 19 11:10:01.339480 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Id: b8f2a78f3a3e4b8e8937e3bc39524690 Jul 19 11:10:01.339480 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Name: tempest-AttachVolumeNegativeTest-434278602-project Jul 19 11:10:01.339480 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: method: > Jul 19 11:10:01.339480 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: exception: Instance 53545a7c-c683-49d6-941d-4445f2b09379 could not be found. Jul 19 11:10:01.339480 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: action_args: {'id': '53545a7c-c683-49d6-941d-4445f2b09379'} Jul 19 11:10:01.340348 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:10:01.340348 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:01.340348 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:10:01.340348 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:10:01.340348 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:10:01.340348 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:10:01.340348 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:10:01.340348 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:10:01.340348 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:10:01.340348 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:10:01.340348 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 53545a7c-c683-49d6-941d-4445f2b09379 could not be found. Jul 19 11:10:01.340348 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:01.340348 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:10:01.340348 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:01.340348 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:01.340348 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:10:01.340348 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:10:01.340348 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:10:01.340348 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:10:01.340348 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:10:01.341771 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:10:01.341771 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 19 11:10:01.341771 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 19 11:10:01.341771 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:10:01.341771 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:10:01.341771 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:10:01.341771 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:10:01.341771 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 53545a7c-c683-49d6-941d-4445f2b09379 could not be found. Jul 19 11:10:01.341771 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:01.341771 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:10:01.341771 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:01.341771 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:01.341771 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:10:01.341771 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:10:01.341771 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:10:01.341771 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:10:01.341771 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 53545a7c-c683-49d6-941d-4445f2b09379 could not be found. Jul 19 11:10:01.341771 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:01.343115 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-d80439ee-8e70-422a-89ff-743bdab37f17 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Returning 404 to user: Instance 53545a7c-c683-49d6-941d-4445f2b09379 could not be found. {{(pid=117849) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:10:01.343115 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-d80439ee-8e70-422a-89ff-743bdab37f17 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379" status: 404 len: 111 microversion: 2.1 time: 0.029407 Jul 19 11:10:01.343115 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 414/849] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:01 2022] GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:10:01.348233 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-e7dae014-67a9-4d30-99f2-509aa1f0230e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '.version_select at 0x7f04aa4505e0>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:01.365157 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-e7dae014-67a9-4d30-99f2-509aa1f0230e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1830923980" status: 202 len: 0 microversion: 2.1 time: 0.020645 Jul 19 11:10:01.365703 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 436/850] 10.222.0.33 () {58 vars in 1220 bytes} [Tue Jul 19 11:10:01 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1830923980 => generated 0 bytes in 22 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 19 11:10:01.963918 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-88cda15f-d1b3-4ffc-ac46-b9393ff07d93 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Action: 'create', calling method: .version_select at 0x7f04aa1c7790>, body: {"keypair": {"name": "tempest-keypair-609097279", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCuRWyNCN/89bm3aJWAyAtFU+KgGhOWXDFJ8FnpEMCewpUzOYN8G4iOhaHxgxDBJ7MMPECfbEJYeybpaQYQWhOnzbItfSiUEVXx5zx4ZDFcqfASFhuuOY612LIoiAog6qg=="}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:10:01.978175 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-88cda15f-d1b3-4ffc-ac46-b9393ff07d93 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Getting quotas for project ef534f1d06ea415d98857ec7f9e5fb97. Resources: {'key_pairs'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:10:01.982178 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-88cda15f-d1b3-4ffc-ac46-b9393ff07d93 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Getting quotas for user b8f2a78f3a3e4b8e8937e3bc39524690 and project ef534f1d06ea415d98857ec7f9e5fb97. Resources: {'key_pairs'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:10:02.003464 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-88cda15f-d1b3-4ffc-ac46-b9393ff07d93 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.043366 Jul 19 11:10:02.004030 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 415/851] 10.222.0.33 () {60 vars in 1158 bytes} [Tue Jul 19 11:10:01 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:10:02.099851 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-b052a43d-1392-4ece-827c-dfd19c9e5398 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:02.100460 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-b052a43d-1392-4ece-827c-dfd19c9e5398 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:02.106497 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b052a43d-1392-4ece-827c-dfd19c9e5398 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:02.106945 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b052a43d-1392-4ece-827c-dfd19c9e5398 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:02.107711 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b052a43d-1392-4ece-827c-dfd19c9e5398 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:02.108240 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b052a43d-1392-4ece-827c-dfd19c9e5398 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:02.250972 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-b052a43d-1392-4ece-827c-dfd19c9e5398 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:02.255509 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:10:02.261699 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-b052a43d-1392-4ece-827c-dfd19c9e5398 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1574 microversion: 2.1 time: 0.166238 Jul 19 11:10:02.262346 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 437/852] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:02 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1574 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:03.160187 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-620d9162-1285-40bb-92cf-b280b7342ec0 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-119722410", "imageRef": "3085883e-2289-4314-aa47-9c41d4368b02", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "0b1fabad-b81a-4ee7-9fd9-62393ba0d37f"}], "security_groups": [{"name": "tempest-securitygroup--1516314195"}], "key_name": "tempest-keypair-609097279", "user_data": "}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:10:03.261046 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.network.neutron [None req-620d9162-1285-40bb-92cf-b280b7342ec0 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] validate_networks() for [('0b1fabad-b81a-4ee7-9fd9-62393ba0d37f', None, None, None, None, None)] {{(pid=117849) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 19 11:10:03.280939 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-420fdaaf-15c8-47e8-b144-4848e6de885d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:03.281541 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-420fdaaf-15c8-47e8-b144-4848e6de885d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:03.284219 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_api:SELECT=61,nova_api:DELETE=1 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:10:03.290885 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-420fdaaf-15c8-47e8-b144-4848e6de885d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:03.291335 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-420fdaaf-15c8-47e8-b144-4848e6de885d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:03.292096 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-420fdaaf-15c8-47e8-b144-4848e6de885d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:03.292528 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-420fdaaf-15c8-47e8-b144-4848e6de885d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:03.431471 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-420fdaaf-15c8-47e8-b144-4848e6de885d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:03.439285 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-420fdaaf-15c8-47e8-b144-4848e6de885d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1574 microversion: 2.1 time: 0.162752 Jul 19 11:10:03.440001 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 438/853] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:03 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1574 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:03.676348 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-620d9162-1285-40bb-92cf-b280b7342ec0 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Flavor limits 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 19 11:10:03.676810 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-620d9162-1285-40bb-92cf-b280b7342ec0 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Image limits 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 19 11:10:03.677283 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-620d9162-1285-40bb-92cf-b280b7342ec0 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Flavor pref 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 19 11:10:03.677691 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-620d9162-1285-40bb-92cf-b280b7342ec0 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Image pref 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 19 11:10:03.678100 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-620d9162-1285-40bb-92cf-b280b7342ec0 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 19 11:10:03.686756 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-620d9162-1285-40bb-92cf-b280b7342ec0 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:03.687187 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-620d9162-1285-40bb-92cf-b280b7342ec0 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:03.687977 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-620d9162-1285-40bb-92cf-b280b7342ec0 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:03.688412 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-620d9162-1285-40bb-92cf-b280b7342ec0 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:03.704864 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-620d9162-1285-40bb-92cf-b280b7342ec0 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Getting quotas for project ef534f1d06ea415d98857ec7f9e5fb97. Resources: {'cores', 'instances', 'ram'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:10:03.708810 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-620d9162-1285-40bb-92cf-b280b7342ec0 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Getting quotas for user b8f2a78f3a3e4b8e8937e3bc39524690 and project ef534f1d06ea415d98857ec7f9e5fb97. Resources: {'cores', 'instances', 'ram'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:10:03.720058 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-620d9162-1285-40bb-92cf-b280b7342ec0 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Going to run 1 instances... {{(pid=117849) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 19 11:10:03.723408 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-620d9162-1285-40bb-92cf-b280b7342ec0 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='3085883e-2289-4314-aa47-9c41d4368b02',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117849) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 19 11:10:03.770183 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-620d9162-1285-40bb-92cf-b280b7342ec0 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.614742 Jul 19 11:10:03.770824 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 416/854] 10.222.0.33 () {60 vars in 1146 bytes} [Tue Jul 19 11:10:03 2022] POST /compute/v2.1/servers => generated 402 bytes in 616 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 19 11:10:03.780999 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-4a96e699-621e-446d-8871-b1e08610b225 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:03.781719 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-4a96e699-621e-446d-8871-b1e08610b225 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:03.835296 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-4a96e699-621e-446d-8871-b1e08610b225 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:03.841966 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-4a96e699-621e-446d-8871-b1e08610b225 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67" status: 200 len: 1240 microversion: 2.1 time: 0.065118 Jul 19 11:10:03.842535 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 439/855] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:03 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67 => generated 1240 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:04.456960 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-7aa2bfd9-83a2-4af4-8df4-0ad45ac69623 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:04.457930 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-7aa2bfd9-83a2-4af4-8df4-0ad45ac69623 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:04.466718 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7aa2bfd9-83a2-4af4-8df4-0ad45ac69623 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:04.467184 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7aa2bfd9-83a2-4af4-8df4-0ad45ac69623 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:04.468304 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7aa2bfd9-83a2-4af4-8df4-0ad45ac69623 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:04.468772 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7aa2bfd9-83a2-4af4-8df4-0ad45ac69623 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:04.605061 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-7aa2bfd9-83a2-4af4-8df4-0ad45ac69623 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:04.612548 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-7aa2bfd9-83a2-4af4-8df4-0ad45ac69623 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1574 microversion: 2.1 time: 0.159986 Jul 19 11:10:04.613137 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 417/856] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:04 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1574 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:04.858676 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-b5cf4a24-2766-4f9b-87cc-117960275b3c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:04.859188 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-b5cf4a24-2766-4f9b-87cc-117960275b3c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:04.865808 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b5cf4a24-2766-4f9b-87cc-117960275b3c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:04.866259 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b5cf4a24-2766-4f9b-87cc-117960275b3c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:04.867026 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b5cf4a24-2766-4f9b-87cc-117960275b3c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:04.867459 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b5cf4a24-2766-4f9b-87cc-117960275b3c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:04.928639 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-b5cf4a24-2766-4f9b-87cc-117960275b3c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:04.936806 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-b5cf4a24-2766-4f9b-87cc-117960275b3c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67" status: 200 len: 1292 microversion: 2.1 time: 0.082613 Jul 19 11:10:04.937371 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 440/857] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:04 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67 => generated 1292 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:05.629817 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-84e60f40-0187-4cfe-b762-fe6c3e2ac2bc tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:05.630421 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-84e60f40-0187-4cfe-b762-fe6c3e2ac2bc tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:05.637100 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-84e60f40-0187-4cfe-b762-fe6c3e2ac2bc tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:05.637549 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-84e60f40-0187-4cfe-b762-fe6c3e2ac2bc tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:05.638857 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-84e60f40-0187-4cfe-b762-fe6c3e2ac2bc tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:05.639302 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-84e60f40-0187-4cfe-b762-fe6c3e2ac2bc tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:05.860154 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-84e60f40-0187-4cfe-b762-fe6c3e2ac2bc tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:05.867743 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-84e60f40-0187-4cfe-b762-fe6c3e2ac2bc tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1574 microversion: 2.1 time: 0.241771 Jul 19 11:10:05.868380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 418/858] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:05 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1574 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:05.954480 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-883bf7f6-1b06-4d93-8216-3d08cfe33841 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:05.955197 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-883bf7f6-1b06-4d93-8216-3d08cfe33841 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:05.961819 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-883bf7f6-1b06-4d93-8216-3d08cfe33841 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:05.962270 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-883bf7f6-1b06-4d93-8216-3d08cfe33841 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:05.963034 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-883bf7f6-1b06-4d93-8216-3d08cfe33841 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:05.963474 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-883bf7f6-1b06-4d93-8216-3d08cfe33841 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:06.110501 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-883bf7f6-1b06-4d93-8216-3d08cfe33841 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:06.118143 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-883bf7f6-1b06-4d93-8216-3d08cfe33841 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67" status: 200 len: 1366 microversion: 2.1 time: 0.168097 Jul 19 11:10:06.118729 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 441/859] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:05 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67 => generated 1366 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:10:06.631602 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-3b081150-a6ac-4e75-80cd-2bd417938b39 req-7e504a24-4d9e-47dc-aa17-5984d453a7ef service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e9cecc8a-4c06-454a-91b6-0b735a222e67", "tag": "8bc77a5e-bfb6-4334-8a9a-3ebaf41dd8e5"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:10:06.641838 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-3b081150-a6ac-4e75-80cd-2bd417938b39 req-7e504a24-4d9e-47dc-aa17-5984d453a7ef service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:06.642878 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-3b081150-a6ac-4e75-80cd-2bd417938b39 req-7e504a24-4d9e-47dc-aa17-5984d453a7ef service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:06.656331 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-3b081150-a6ac-4e75-80cd-2bd417938b39 req-7e504a24-4d9e-47dc-aa17-5984d453a7ef service nova] Creating event network-changed:8bc77a5e-bfb6-4334-8a9a-3ebaf41dd8e5 for instance e9cecc8a-4c06-454a-91b6-0b735a222e67 on hv-cinder-maste Jul 19 11:10:06.661465 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-3b081150-a6ac-4e75-80cd-2bd417938b39 req-7e504a24-4d9e-47dc-aa17-5984d453a7ef service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033524 Jul 19 11:10:06.661998 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 419/860] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:10:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:10:06.886770 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-416dc2a8-61dd-48e5-8562-1f877678e0d7 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:06.887568 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-416dc2a8-61dd-48e5-8562-1f877678e0d7 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:06.894816 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-416dc2a8-61dd-48e5-8562-1f877678e0d7 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:06.895069 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-416dc2a8-61dd-48e5-8562-1f877678e0d7 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:06.895834 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-416dc2a8-61dd-48e5-8562-1f877678e0d7 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:06.896261 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-416dc2a8-61dd-48e5-8562-1f877678e0d7 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:06.975109 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-b74c6df5-104a-498b-af64-62cffa2ee1e8 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:06.975971 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-b74c6df5-104a-498b-af64-62cffa2ee1e8 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:06.983028 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b74c6df5-104a-498b-af64-62cffa2ee1e8 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:06.983803 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b74c6df5-104a-498b-af64-62cffa2ee1e8 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:06.984571 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b74c6df5-104a-498b-af64-62cffa2ee1e8 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:06.985202 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b74c6df5-104a-498b-af64-62cffa2ee1e8 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:07.065001 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-416dc2a8-61dd-48e5-8562-1f877678e0d7 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:07.072613 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-416dc2a8-61dd-48e5-8562-1f877678e0d7 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1574 microversion: 2.1 time: 0.190204 Jul 19 11:10:07.073219 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 442/861] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:06 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1574 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:07.074102 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.objects.instance [None req-b74c6df5-104a-498b-af64-62cffa2ee1e8 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lazy-loading 'services' on Instance uuid b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 {{(pid=117849) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 19 11:10:07.139251 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-1744f7e7-fba4-4cf0-9d8f-84afd9e96c07 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:07.140107 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-1744f7e7-fba4-4cf0-9d8f-84afd9e96c07 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:07.146783 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1744f7e7-fba4-4cf0-9d8f-84afd9e96c07 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:07.147293 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1744f7e7-fba4-4cf0-9d8f-84afd9e96c07 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:07.148198 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1744f7e7-fba4-4cf0-9d8f-84afd9e96c07 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:07.148757 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1744f7e7-fba4-4cf0-9d8f-84afd9e96c07 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:07.153236 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-b74c6df5-104a-498b-af64-62cffa2ee1e8 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "DELETE /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3/os-volume_attachments/5ac37faf-6388-4d0e-9939-097bb5b52afc" status: 202 len: 0 microversion: 2.1 time: 0.184157 Jul 19 11:10:07.154499 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 420/862] 10.222.0.33 () {58 vars in 1415 bytes} [Tue Jul 19 11:10:06 2022] DELETE /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3/os-volume_attachments/5ac37faf-6388-4d0e-9939-097bb5b52afc => generated 0 bytes in 187 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 19 11:10:07.163340 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-dc15a22a-1e87-4f35-956e-ec3d17b0098c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:07.164497 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-dc15a22a-1e87-4f35-956e-ec3d17b0098c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:07.170942 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dc15a22a-1e87-4f35-956e-ec3d17b0098c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:07.171426 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dc15a22a-1e87-4f35-956e-ec3d17b0098c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:07.172255 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dc15a22a-1e87-4f35-956e-ec3d17b0098c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:07.172742 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dc15a22a-1e87-4f35-956e-ec3d17b0098c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:07.209075 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-dc15a22a-1e87-4f35-956e-ec3d17b0098c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049968 Jul 19 11:10:07.209641 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 421/863] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:10:07 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:10:07.282715 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-1744f7e7-fba4-4cf0-9d8f-84afd9e96c07 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:07.290159 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-1744f7e7-fba4-4cf0-9d8f-84afd9e96c07 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67" status: 200 len: 1366 microversion: 2.1 time: 0.158264 Jul 19 11:10:07.290722 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 443/864] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:07 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67 => generated 1366 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:10:08.090265 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-0c3222ca-5a9e-42ab-b312-c51ec41f3376 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:08.091394 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-0c3222ca-5a9e-42ab-b312-c51ec41f3376 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:08.097983 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0c3222ca-5a9e-42ab-b312-c51ec41f3376 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:08.098434 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0c3222ca-5a9e-42ab-b312-c51ec41f3376 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:08.099199 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0c3222ca-5a9e-42ab-b312-c51ec41f3376 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:08.099629 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0c3222ca-5a9e-42ab-b312-c51ec41f3376 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:08.145907 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-8c3c927a-476a-49df-be5e-bd7ecc996202 req-317868bd-8555-4652-8445-ce74e6b6bc69 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5b72a14b-bd18-4dfa-a68f-12a5424f471c", "name": "network-vif-plugged", "status": "completed", "tag": "da843419-4401-4a26-82c2-7930b51aa1b8"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:10:08.156350 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-8c3c927a-476a-49df-be5e-bd7ecc996202 req-317868bd-8555-4652-8445-ce74e6b6bc69 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:08.156939 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-8c3c927a-476a-49df-be5e-bd7ecc996202 req-317868bd-8555-4652-8445-ce74e6b6bc69 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:08.168495 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-8c3c927a-476a-49df-be5e-bd7ecc996202 req-317868bd-8555-4652-8445-ce74e6b6bc69 service nova] Creating event network-vif-plugged:da843419-4401-4a26-82c2-7930b51aa1b8 for instance 5b72a14b-bd18-4dfa-a68f-12a5424f471c on hv-cinder-maste Jul 19 11:10:08.174092 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-8c3c927a-476a-49df-be5e-bd7ecc996202 req-317868bd-8555-4652-8445-ce74e6b6bc69 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031882 Jul 19 11:10:08.174779 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 444/865] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:10:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:10:08.220862 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-8e663219-4f7d-4cc0-a560-debefc230bc4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:08.221860 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-8e663219-4f7d-4cc0-a560-debefc230bc4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:08.227977 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8e663219-4f7d-4cc0-a560-debefc230bc4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:08.228561 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8e663219-4f7d-4cc0-a560-debefc230bc4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:08.229346 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8e663219-4f7d-4cc0-a560-debefc230bc4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:08.229788 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8e663219-4f7d-4cc0-a560-debefc230bc4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:08.237243 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-0c3222ca-5a9e-42ab-b312-c51ec41f3376 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:08.244797 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-0c3222ca-5a9e-42ab-b312-c51ec41f3376 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1574 microversion: 2.1 time: 0.158325 Jul 19 11:10:08.245359 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 422/866] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:08 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1574 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:08.259802 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-8e663219-4f7d-4cc0-a560-debefc230bc4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043743 Jul 19 11:10:08.260345 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 445/867] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:10:08 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:10:08.305877 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-f8c1799f-cafe-43e6-a144-dc3fee2d1301 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:08.306420 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-f8c1799f-cafe-43e6-a144-dc3fee2d1301 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:08.312361 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f8c1799f-cafe-43e6-a144-dc3fee2d1301 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:08.312848 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f8c1799f-cafe-43e6-a144-dc3fee2d1301 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:08.313608 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f8c1799f-cafe-43e6-a144-dc3fee2d1301 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:08.314042 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f8c1799f-cafe-43e6-a144-dc3fee2d1301 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:08.719379 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-f8c1799f-cafe-43e6-a144-dc3fee2d1301 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:08.726799 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-f8c1799f-cafe-43e6-a144-dc3fee2d1301 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67" status: 200 len: 1366 microversion: 2.1 time: 0.424769 Jul 19 11:10:08.727356 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 423/868] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:08 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67 => generated 1366 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:09.263406 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-8ce3f66b-d16f-47da-abbe-7cea9da5f4ec tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:09.264295 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-8ce3f66b-d16f-47da-abbe-7cea9da5f4ec tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:09.272355 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-52b4f1a6-2f76-49bf-9525-707d7c4b1d58 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:09.272999 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8ce3f66b-d16f-47da-abbe-7cea9da5f4ec tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:09.273516 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8ce3f66b-d16f-47da-abbe-7cea9da5f4ec tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:09.273629 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-52b4f1a6-2f76-49bf-9525-707d7c4b1d58 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:09.274281 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8ce3f66b-d16f-47da-abbe-7cea9da5f4ec tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:09.274711 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8ce3f66b-d16f-47da-abbe-7cea9da5f4ec tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:09.281447 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-52b4f1a6-2f76-49bf-9525-707d7c4b1d58 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:09.282106 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-52b4f1a6-2f76-49bf-9525-707d7c4b1d58 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:09.282880 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-52b4f1a6-2f76-49bf-9525-707d7c4b1d58 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:09.283435 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-52b4f1a6-2f76-49bf-9525-707d7c4b1d58 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:09.312942 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-52b4f1a6-2f76-49bf-9525-707d7c4b1d58 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045232 Jul 19 11:10:09.313510 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 424/869] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:10:09 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:10:09.407366 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-8ce3f66b-d16f-47da-abbe-7cea9da5f4ec tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:09.414731 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-8ce3f66b-d16f-47da-abbe-7cea9da5f4ec tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1574 microversion: 2.1 time: 0.156405 Jul 19 11:10:09.415287 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 446/870] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:09 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1574 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:09.743892 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-feb3cdde-16f4-4f1e-a6aa-cff32807d157 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:09.744611 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-feb3cdde-16f4-4f1e-a6aa-cff32807d157 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:09.751071 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-feb3cdde-16f4-4f1e-a6aa-cff32807d157 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:09.751512 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-feb3cdde-16f4-4f1e-a6aa-cff32807d157 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:09.752258 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-feb3cdde-16f4-4f1e-a6aa-cff32807d157 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:09.752714 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-feb3cdde-16f4-4f1e-a6aa-cff32807d157 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:09.889486 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-feb3cdde-16f4-4f1e-a6aa-cff32807d157 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:09.896926 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-feb3cdde-16f4-4f1e-a6aa-cff32807d157 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67" status: 200 len: 1366 microversion: 2.1 time: 0.156825 Jul 19 11:10:09.897501 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 425/871] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:09 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67 => generated 1366 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:10.323697 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-37bda401-f8a6-4704-b27a-3af3e944fc7f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:10.324616 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-37bda401-f8a6-4704-b27a-3af3e944fc7f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:10.332080 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-37bda401-f8a6-4704-b27a-3af3e944fc7f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:10.332532 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-37bda401-f8a6-4704-b27a-3af3e944fc7f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:10.333334 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-37bda401-f8a6-4704-b27a-3af3e944fc7f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:10.333767 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-37bda401-f8a6-4704-b27a-3af3e944fc7f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:10.362793 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-37bda401-f8a6-4704-b27a-3af3e944fc7f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.043086 Jul 19 11:10:10.363349 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 447/872] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:10:10 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3/os-volume_attachments => generated 25 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 19 11:10:10.432656 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-3043e3e2-a9b3-46f2-b806-cb5a17881885 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:10.433159 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-3043e3e2-a9b3-46f2-b806-cb5a17881885 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:10.434067 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-9a60e94d-0f63-459b-8314-c8511de416e7 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:10.435679 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-9a60e94d-0f63-459b-8314-c8511de416e7 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:10.441284 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3043e3e2-a9b3-46f2-b806-cb5a17881885 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:10.441733 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3043e3e2-a9b3-46f2-b806-cb5a17881885 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:10.442498 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3043e3e2-a9b3-46f2-b806-cb5a17881885 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:10.442935 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3043e3e2-a9b3-46f2-b806-cb5a17881885 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:10.443081 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9a60e94d-0f63-459b-8314-c8511de416e7 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:10.443671 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9a60e94d-0f63-459b-8314-c8511de416e7 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:10.444586 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9a60e94d-0f63-459b-8314-c8511de416e7 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:10.445193 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9a60e94d-0f63-459b-8314-c8511de416e7 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:10.468165 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-3043e3e2-a9b3-46f2-b806-cb5a17881885 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Going to try to terminate instance {{(pid=117848) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 19 11:10:10.556156 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-3043e3e2-a9b3-46f2-b806-cb5a17881885 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "DELETE /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 204 len: 0 microversion: 2.1 time: 0.124859 Jul 19 11:10:10.556767 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 448/873] 10.222.0.33 () {58 vars in 1238 bytes} [Tue Jul 19 11:10:10 2022] DELETE /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 19 11:10:10.564983 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-066bd253-98b6-4ea2-a296-1dbd1f098755 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:10.565680 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-066bd253-98b6-4ea2-a296-1dbd1f098755 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:10.571547 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-066bd253-98b6-4ea2-a296-1dbd1f098755 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:10.572000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-066bd253-98b6-4ea2-a296-1dbd1f098755 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:10.572801 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-066bd253-98b6-4ea2-a296-1dbd1f098755 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:10.573241 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-066bd253-98b6-4ea2-a296-1dbd1f098755 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:10.589774 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-9a60e94d-0f63-459b-8314-c8511de416e7 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:10.598012 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-9a60e94d-0f63-459b-8314-c8511de416e7 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1574 microversion: 2.1 time: 0.169558 Jul 19 11:10:10.598582 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 426/874] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:10 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1574 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:10.707130 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-066bd253-98b6-4ea2-a296-1dbd1f098755 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:10.714847 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-066bd253-98b6-4ea2-a296-1dbd1f098755 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1689 microversion: 2.1 time: 0.153932 Jul 19 11:10:10.715422 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 449/875] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:10 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1689 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:10:10.913565 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-84017083-e11b-448c-a7ad-7accdb2bacf6 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:10.914078 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-84017083-e11b-448c-a7ad-7accdb2bacf6 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:10.920289 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-84017083-e11b-448c-a7ad-7accdb2bacf6 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:10.920730 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-84017083-e11b-448c-a7ad-7accdb2bacf6 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:10.921488 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-84017083-e11b-448c-a7ad-7accdb2bacf6 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:10.921918 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-84017083-e11b-448c-a7ad-7accdb2bacf6 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:11.046397 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-84017083-e11b-448c-a7ad-7accdb2bacf6 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:11.053987 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-84017083-e11b-448c-a7ad-7accdb2bacf6 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67" status: 200 len: 1366 microversion: 2.1 time: 0.144274 Jul 19 11:10:11.054566 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 427/876] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:10 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67 => generated 1366 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:11.614729 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-9b0bdd62-73e2-4e87-9486-c1d16aa21b1a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:11.615250 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-9b0bdd62-73e2-4e87-9486-c1d16aa21b1a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:11.622038 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9b0bdd62-73e2-4e87-9486-c1d16aa21b1a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:11.622483 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9b0bdd62-73e2-4e87-9486-c1d16aa21b1a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:11.623231 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9b0bdd62-73e2-4e87-9486-c1d16aa21b1a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:11.623659 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9b0bdd62-73e2-4e87-9486-c1d16aa21b1a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:11.732561 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-bc8c5e7b-6af0-46eb-afc4-46c0f7312435 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:11.733281 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-bc8c5e7b-6af0-46eb-afc4-46c0f7312435 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:11.735634 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_api:SELECT=95,nova_api:UPDATE=1,nova_api:INSERT=4 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:10:11.740983 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bc8c5e7b-6af0-46eb-afc4-46c0f7312435 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:11.741439 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bc8c5e7b-6af0-46eb-afc4-46c0f7312435 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:11.742207 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bc8c5e7b-6af0-46eb-afc4-46c0f7312435 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:11.742641 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bc8c5e7b-6af0-46eb-afc4-46c0f7312435 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:11.755797 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-9b0bdd62-73e2-4e87-9486-c1d16aa21b1a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:11.764090 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-9b0bdd62-73e2-4e87-9486-c1d16aa21b1a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1561 microversion: 2.1 time: 0.153486 Jul 19 11:10:11.764785 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 450/877] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:11 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:11.871219 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-bc8c5e7b-6af0-46eb-afc4-46c0f7312435 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:11.879370 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-bc8c5e7b-6af0-46eb-afc4-46c0f7312435 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1689 microversion: 2.1 time: 0.150728 Jul 19 11:10:11.879933 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 428/878] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:11 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1689 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:10:12.070904 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-736d45dd-8545-4fb4-bd6c-aa52313e1f2d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:12.071548 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-736d45dd-8545-4fb4-bd6c-aa52313e1f2d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:12.078275 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-736d45dd-8545-4fb4-bd6c-aa52313e1f2d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:12.078730 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-736d45dd-8545-4fb4-bd6c-aa52313e1f2d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:12.079496 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-736d45dd-8545-4fb4-bd6c-aa52313e1f2d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:12.079934 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-736d45dd-8545-4fb4-bd6c-aa52313e1f2d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:12.214582 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-736d45dd-8545-4fb4-bd6c-aa52313e1f2d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:12.222369 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-736d45dd-8545-4fb4-bd6c-aa52313e1f2d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67" status: 200 len: 1366 microversion: 2.1 time: 0.155243 Jul 19 11:10:12.223081 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 451/879] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:12 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67 => generated 1366 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:12.224809 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-fe437955-4a10-4e60-a595-048e9fd8fe7c req-e2ec3473-3b35-4a1c-8113-f1c947a58bda service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e9cecc8a-4c06-454a-91b6-0b735a222e67", "name": "network-vif-plugged", "status": "completed", "tag": "8bc77a5e-bfb6-4334-8a9a-3ebaf41dd8e5"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:10:12.235243 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-fe437955-4a10-4e60-a595-048e9fd8fe7c req-e2ec3473-3b35-4a1c-8113-f1c947a58bda service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:12.235681 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-fe437955-4a10-4e60-a595-048e9fd8fe7c req-e2ec3473-3b35-4a1c-8113-f1c947a58bda service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:12.246676 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-fe437955-4a10-4e60-a595-048e9fd8fe7c req-e2ec3473-3b35-4a1c-8113-f1c947a58bda service nova] Creating event network-vif-plugged:8bc77a5e-bfb6-4334-8a9a-3ebaf41dd8e5 for instance e9cecc8a-4c06-454a-91b6-0b735a222e67 on hv-cinder-maste Jul 19 11:10:12.252421 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-fe437955-4a10-4e60-a595-048e9fd8fe7c req-e2ec3473-3b35-4a1c-8113-f1c947a58bda service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.095176 Jul 19 11:10:12.253016 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 429/880] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:10:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:10:12.896859 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-64e2b4cf-068b-4d07-ab65-901844e3d895 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:12.897458 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-64e2b4cf-068b-4d07-ab65-901844e3d895 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:12.903944 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-64e2b4cf-068b-4d07-ab65-901844e3d895 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:12.904396 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-64e2b4cf-068b-4d07-ab65-901844e3d895 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:12.905209 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-64e2b4cf-068b-4d07-ab65-901844e3d895 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:12.905642 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-64e2b4cf-068b-4d07-ab65-901844e3d895 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:13.033707 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-64e2b4cf-068b-4d07-ab65-901844e3d895 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:13.041098 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-64e2b4cf-068b-4d07-ab65-901844e3d895 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1689 microversion: 2.1 time: 0.148149 Jul 19 11:10:13.041666 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 452/881] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:12 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1689 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:13.239876 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-c4af85dd-bc0e-4d96-99ee-4efd093e5146 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:13.240568 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-c4af85dd-bc0e-4d96-99ee-4efd093e5146 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:13.247468 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c4af85dd-bc0e-4d96-99ee-4efd093e5146 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:13.248416 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c4af85dd-bc0e-4d96-99ee-4efd093e5146 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:13.249462 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c4af85dd-bc0e-4d96-99ee-4efd093e5146 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:13.249896 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c4af85dd-bc0e-4d96-99ee-4efd093e5146 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:13.377868 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-c4af85dd-bc0e-4d96-99ee-4efd093e5146 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:13.385599 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-c4af85dd-bc0e-4d96-99ee-4efd093e5146 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67" status: 200 len: 1366 microversion: 2.1 time: 0.149891 Jul 19 11:10:13.386160 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 430/882] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:13 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67 => generated 1366 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:14.057870 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-b69d4975-8f45-4e1d-bb1f-ebd706f61143 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:14.058471 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-b69d4975-8f45-4e1d-bb1f-ebd706f61143 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:14.060426 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell0:SELECT=2 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:10:14.066029 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b69d4975-8f45-4e1d-bb1f-ebd706f61143 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:14.066476 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b69d4975-8f45-4e1d-bb1f-ebd706f61143 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:14.067233 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b69d4975-8f45-4e1d-bb1f-ebd706f61143 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:14.067665 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b69d4975-8f45-4e1d-bb1f-ebd706f61143 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:14.193160 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-b69d4975-8f45-4e1d-bb1f-ebd706f61143 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:14.200521 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-b69d4975-8f45-4e1d-bb1f-ebd706f61143 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1689 microversion: 2.1 time: 0.146499 Jul 19 11:10:14.201119 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 453/883] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:14 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1689 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:14.401969 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-39ffd0c9-3156-48cc-be2f-8ad27be6919c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:14.402499 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-39ffd0c9-3156-48cc-be2f-8ad27be6919c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:14.405248 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell0:SELECT=2 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:10:14.406053 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell0:SELECT=10 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:10:14.412638 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-39ffd0c9-3156-48cc-be2f-8ad27be6919c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:14.413139 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-39ffd0c9-3156-48cc-be2f-8ad27be6919c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:14.413911 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-39ffd0c9-3156-48cc-be2f-8ad27be6919c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:14.414340 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-39ffd0c9-3156-48cc-be2f-8ad27be6919c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:14.544147 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-39ffd0c9-3156-48cc-be2f-8ad27be6919c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:14.547526 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:10:14.552315 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-39ffd0c9-3156-48cc-be2f-8ad27be6919c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67" status: 200 len: 1366 microversion: 2.1 time: 0.154252 Jul 19 11:10:14.552911 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 431/884] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:14 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67 => generated 1366 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:15.217392 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-ab1f635f-9eea-4a7d-8586-5b1f91e2ec9e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:15.217996 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-ab1f635f-9eea-4a7d-8586-5b1f91e2ec9e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:15.224484 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ab1f635f-9eea-4a7d-8586-5b1f91e2ec9e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:15.225043 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ab1f635f-9eea-4a7d-8586-5b1f91e2ec9e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:15.225807 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ab1f635f-9eea-4a7d-8586-5b1f91e2ec9e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:15.226242 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ab1f635f-9eea-4a7d-8586-5b1f91e2ec9e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:15.245738 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:10:15.569728 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-cbe526bc-1dfa-49c1-91a0-1938842a6574 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:15.570512 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-cbe526bc-1dfa-49c1-91a0-1938842a6574 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:15.577257 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cbe526bc-1dfa-49c1-91a0-1938842a6574 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:15.577709 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cbe526bc-1dfa-49c1-91a0-1938842a6574 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:15.578472 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cbe526bc-1dfa-49c1-91a0-1938842a6574 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:15.578905 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cbe526bc-1dfa-49c1-91a0-1938842a6574 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:15.652293 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-ab1f635f-9eea-4a7d-8586-5b1f91e2ec9e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:15.660330 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-ab1f635f-9eea-4a7d-8586-5b1f91e2ec9e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1689 microversion: 2.1 time: 0.446857 Jul 19 11:10:15.660922 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 454/885] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:15 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1689 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:15.813127 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-cbe526bc-1dfa-49c1-91a0-1938842a6574 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:15.823379 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-cbe526bc-1dfa-49c1-91a0-1938842a6574 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67" status: 200 len: 1366 microversion: 2.1 time: 0.258040 Jul 19 11:10:15.823964 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 432/886] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:15 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67 => generated 1366 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:10:16.167160 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-0d9a2a0c-4cfa-4d01-afeb-5559a6e8fb43 req-a28fa9e4-e8a6-42d8-9e19-5b6585d7d462 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b9897cfb-9ec6-41f4-bcb5-ba1a978421d3", "name": "network-vif-deleted", "tag": "bee94173-a17a-45b4-ab19-e862338cf16b"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:10:16.175074 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-0d9a2a0c-4cfa-4d01-afeb-5559a6e8fb43 req-a28fa9e4-e8a6-42d8-9e19-5b6585d7d462 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:16.175517 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-0d9a2a0c-4cfa-4d01-afeb-5559a6e8fb43 req-a28fa9e4-e8a6-42d8-9e19-5b6585d7d462 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:16.186818 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-0d9a2a0c-4cfa-4d01-afeb-5559a6e8fb43 req-a28fa9e4-e8a6-42d8-9e19-5b6585d7d462 service nova] Creating event network-vif-deleted:bee94173-a17a-45b4-ab19-e862338cf16b for instance b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 on hv-cinder-maste Jul 19 11:10:16.191809 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-0d9a2a0c-4cfa-4d01-afeb-5559a6e8fb43 req-a28fa9e4-e8a6-42d8-9e19-5b6585d7d462 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.028497 Jul 19 11:10:16.192385 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 455/887] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:10:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:10:16.677663 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-1b780df5-d398-4f4a-a235-b9e5b70d2b02 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:16.678274 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-1b780df5-d398-4f4a-a235-b9e5b70d2b02 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:16.685128 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1b780df5-d398-4f4a-a235-b9e5b70d2b02 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:16.686174 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1b780df5-d398-4f4a-a235-b9e5b70d2b02 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:16.686944 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1b780df5-d398-4f4a-a235-b9e5b70d2b02 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:16.687384 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1b780df5-d398-4f4a-a235-b9e5b70d2b02 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:16.717819 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1b780df5-d398-4f4a-a235-b9e5b70d2b02 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:16.718531 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1b780df5-d398-4f4a-a235-b9e5b70d2b02 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:16.718978 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.objects.instance [None req-1b780df5-d398-4f4a-a235-b9e5b70d2b02 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lazy-loading 'fault' on Instance uuid b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 {{(pid=117849) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 19 11:10:16.768623 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-1b780df5-d398-4f4a-a235-b9e5b70d2b02 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:16.776122 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-1b780df5-d398-4f4a-a235-b9e5b70d2b02 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1335 microversion: 2.1 time: 0.102386 Jul 19 11:10:16.776684 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 433/888] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:16 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1335 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:16.842093 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-9aa61ece-90dc-4978-9a25-545ca2e21fed tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:16.842960 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-9aa61ece-90dc-4978-9a25-545ca2e21fed tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:16.849374 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9aa61ece-90dc-4978-9a25-545ca2e21fed tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:16.849824 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9aa61ece-90dc-4978-9a25-545ca2e21fed tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:16.850596 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9aa61ece-90dc-4978-9a25-545ca2e21fed tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:16.851037 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9aa61ece-90dc-4978-9a25-545ca2e21fed tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:16.992080 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-9aa61ece-90dc-4978-9a25-545ca2e21fed tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:17.001944 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-9aa61ece-90dc-4978-9a25-545ca2e21fed tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67" status: 200 len: 1366 microversion: 2.1 time: 0.164661 Jul 19 11:10:17.002771 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 456/889] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:16 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67 => generated 1366 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:17.792037 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-f66266d6-8764-4ab3-a263-4bfb4851bc71 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:17.792651 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-f66266d6-8764-4ab3-a263-4bfb4851bc71 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:17.800017 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f66266d6-8764-4ab3-a263-4bfb4851bc71 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:17.800478 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f66266d6-8764-4ab3-a263-4bfb4851bc71 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:17.801274 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f66266d6-8764-4ab3-a263-4bfb4851bc71 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:17.801801 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f66266d6-8764-4ab3-a263-4bfb4851bc71 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:17.833450 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f66266d6-8764-4ab3-a263-4bfb4851bc71 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:17.833901 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f66266d6-8764-4ab3-a263-4bfb4851bc71 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:17.835427 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.objects.instance [None req-f66266d6-8764-4ab3-a263-4bfb4851bc71 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lazy-loading 'fault' on Instance uuid b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 {{(pid=117849) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 19 11:10:17.878961 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-f66266d6-8764-4ab3-a263-4bfb4851bc71 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:17.886720 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-f66266d6-8764-4ab3-a263-4bfb4851bc71 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 200 len: 1335 microversion: 2.1 time: 0.098484 Jul 19 11:10:17.887630 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 434/890] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:17 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 1335 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:18.019015 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-c188fb67-7f9a-4d35-8043-bfeef8583812 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:18.019554 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-c188fb67-7f9a-4d35-8043-bfeef8583812 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:18.026028 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c188fb67-7f9a-4d35-8043-bfeef8583812 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:18.026483 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c188fb67-7f9a-4d35-8043-bfeef8583812 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:18.027240 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c188fb67-7f9a-4d35-8043-bfeef8583812 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:18.027676 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c188fb67-7f9a-4d35-8043-bfeef8583812 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:18.161405 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-c188fb67-7f9a-4d35-8043-bfeef8583812 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:18.168822 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-c188fb67-7f9a-4d35-8043-bfeef8583812 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67" status: 200 len: 1556 microversion: 2.1 time: 0.153805 Jul 19 11:10:18.169390 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 457/891] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:18 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67 => generated 1556 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:18.187270 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-9bfac561-6a7a-494f-b369-7961e937d2dd tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:18.187940 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-9bfac561-6a7a-494f-b369-7961e937d2dd tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:18.195168 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9bfac561-6a7a-494f-b369-7961e937d2dd tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:18.195824 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9bfac561-6a7a-494f-b369-7961e937d2dd tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:18.196994 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9bfac561-6a7a-494f-b369-7961e937d2dd tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:18.197635 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9bfac561-6a7a-494f-b369-7961e937d2dd tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:18.277442 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-9bfac561-6a7a-494f-b369-7961e937d2dd tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.096290 Jul 19 11:10:18.278024 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 435/892] 10.222.0.33 () {58 vars in 1274 bytes} [Tue Jul 19 11:10:18 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-interface => generated 284 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:10:18.903617 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-53cfcc6e-dd27-4761-8718-c515f2f6aa7e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:18.904216 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-53cfcc6e-dd27-4761-8718-c515f2f6aa7e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: b9897cfb-9ec6-41f4-bcb5-ba1a978421d3] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:18.910332 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-53cfcc6e-dd27-4761-8718-c515f2f6aa7e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:18.910775 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-53cfcc6e-dd27-4761-8718-c515f2f6aa7e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:18.911530 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-53cfcc6e-dd27-4761-8718-c515f2f6aa7e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:18.911958 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-53cfcc6e-dd27-4761-8718-c515f2f6aa7e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:18.926136 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.wsgi [None req-53cfcc6e-dd27-4761-8718-c515f2f6aa7e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] HTTP exception thrown: Instance b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 could not be found. Jul 19 11:10:18.927848 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-53cfcc6e-dd27-4761-8718-c515f2f6aa7e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Request method failure captured: Jul 19 11:10:18.927848 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: request: GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 HTTP/1.1 Jul 19 11:10:18.927848 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept: application/json Jul 19 11:10:18.927848 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept-Encoding: identity Jul 19 11:10:18.927848 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Connection: close Jul 19 11:10:18.927848 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Length: 0 Jul 19 11:10:18.927848 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Type: application/json Jul 19 11:10:18.927848 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Host: 10.222.0.33 Jul 19 11:10:18.927848 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Openstack-System-Scope: None Jul 19 11:10:18.927848 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: User-Agent: python-urllib3/1.26.9 Jul 19 11:10:18.927848 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Auth-Token: gAAAAABi1o_D_huvBlD9ZJOIcq8Z1aOSnILmJp8MQgaQ8MuWsEAL_ZmZPsH--WyF20ahgvN08GveS7vk1bvyoVIPWYbHaxCp791BKfyC6yZ04rwPPH3vQlTPlGO54x_Ovlr-nuJTMqDQHYocrhpCc4nPGpje8DwiZIJCWR_YJdRwNKP8_4bwd5o Jul 19 11:10:18.927848 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Id: None Jul 19 11:10:18.927848 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Name: None Jul 19 11:10:18.927848 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Identity-Status: Confirmed Jul 19 11:10:18.927848 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Is-Admin-Project: True Jul 19 11:10:18.927848 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Id: default Jul 19 11:10:18.927848 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Name: Default Jul 19 11:10:18.927848 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Id: 05fceba2c70c40a399e71daabe8edef6 Jul 19 11:10:18.927848 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1654338802 Jul 19 11:10:18.927848 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Role: member,reader Jul 19 11:10:18.927848 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Roles: member,reader Jul 19 11:10:18.929238 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/05fceba2c70c40a399e71daabe8edef6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/05fceba2c70c40a399e71daabe8edef6"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/05fceba2c70c40a399e71daabe8edef6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_05fceba2c70c40a399e71daabe8edef6"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:10:18.929238 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1654338802 Jul 19 11:10:18.929238 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Id: 05fceba2c70c40a399e71daabe8edef6 Jul 19 11:10:18.929238 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1654338802 Jul 19 11:10:18.929238 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User: tempest-ServerRescueNegativeTestJSON-1654338802-project Jul 19 11:10:18.929238 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Id: default Jul 19 11:10:18.929238 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Name: Default Jul 19 11:10:18.929238 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Id: c95281b28eae450d8678583a929c3b8a Jul 19 11:10:18.929238 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1654338802-project Jul 19 11:10:18.929238 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: method: > Jul 19 11:10:18.929238 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: exception: Instance b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 could not be found. Jul 19 11:10:18.929238 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: action_args: {'id': 'b9897cfb-9ec6-41f4-bcb5-ba1a978421d3'} Jul 19 11:10:18.929948 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:10:18.929948 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:18.929948 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:10:18.929948 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:10:18.929948 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:10:18.929948 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:10:18.929948 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:10:18.929948 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:10:18.929948 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:10:18.929948 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:10:18.929948 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 could not be found. Jul 19 11:10:18.929948 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:18.929948 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:10:18.929948 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:18.929948 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:18.929948 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:10:18.929948 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:10:18.929948 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:10:18.929948 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:10:18.929948 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:10:18.931095 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:10:18.931095 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 19 11:10:18.931095 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 19 11:10:18.931095 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:10:18.931095 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:10:18.931095 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:10:18.931095 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:10:18.931095 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 could not be found. Jul 19 11:10:18.931095 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:18.931095 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:10:18.931095 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:18.931095 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:18.931095 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:10:18.931095 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:10:18.931095 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:10:18.931095 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:10:18.931095 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 could not be found. Jul 19 11:10:18.931095 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:18.932126 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-53cfcc6e-dd27-4761-8718-c515f2f6aa7e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Returning 404 to user: Instance b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 could not be found. {{(pid=117848) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:10:18.932126 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-53cfcc6e-dd27-4761-8718-c515f2f6aa7e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3" status: 404 len: 111 microversion: 2.1 time: 0.029671 Jul 19 11:10:18.933245 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 458/893] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:18 2022] GET /compute/v2.1/servers/b9897cfb-9ec6-41f4-bcb5-ba1a978421d3 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:10:18.940957 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-590fb222-e4f1-44b0-93d4-74bbce20325c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '.version_select at 0x7f04aa158ca0>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:18.970459 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-590fb222-e4f1-44b0-93d4-74bbce20325c tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1183558961" status: 202 len: 0 microversion: 2.1 time: 0.027333 Jul 19 11:10:18.970459 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 436/894] 10.222.0.33 () {58 vars in 1220 bytes} [Tue Jul 19 11:10:18 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1183558961 => generated 0 bytes in 29 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 19 11:10:19.230211 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-51d7ba30-6e08-4235-8b1a-c82c888bb04b req-a8d75037-a522-4cf9-8e12-529c05a8be30 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e9cecc8a-4c06-454a-91b6-0b735a222e67", "tag": "8bc77a5e-bfb6-4334-8a9a-3ebaf41dd8e5"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:10:19.239348 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-51d7ba30-6e08-4235-8b1a-c82c888bb04b req-a8d75037-a522-4cf9-8e12-529c05a8be30 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:19.240091 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-51d7ba30-6e08-4235-8b1a-c82c888bb04b req-a8d75037-a522-4cf9-8e12-529c05a8be30 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:19.252080 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-51d7ba30-6e08-4235-8b1a-c82c888bb04b req-a8d75037-a522-4cf9-8e12-529c05a8be30 service nova] Creating event network-changed:8bc77a5e-bfb6-4334-8a9a-3ebaf41dd8e5 for instance e9cecc8a-4c06-454a-91b6-0b735a222e67 on hv-cinder-maste Jul 19 11:10:19.259033 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-51d7ba30-6e08-4235-8b1a-c82c888bb04b req-a8d75037-a522-4cf9-8e12-529c05a8be30 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032341 Jul 19 11:10:19.260032 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 459/895] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:10:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:10:21.907685 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-fc392562-b904-4a7f-86d7-5d990125e38e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:21.908174 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-fc392562-b904-4a7f-86d7-5d990125e38e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:21.914187 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fc392562-b904-4a7f-86d7-5d990125e38e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:21.914636 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fc392562-b904-4a7f-86d7-5d990125e38e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:21.915394 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fc392562-b904-4a7f-86d7-5d990125e38e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:21.915829 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fc392562-b904-4a7f-86d7-5d990125e38e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:21.942321 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-fc392562-b904-4a7f-86d7-5d990125e38e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Going to try to terminate instance {{(pid=117849) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 19 11:10:22.014864 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-fc392562-b904-4a7f-86d7-5d990125e38e tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "DELETE /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 204 len: 0 microversion: 2.1 time: 0.111409 Jul 19 11:10:22.015433 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 437/896] 10.222.0.33 () {58 vars in 1238 bytes} [Tue Jul 19 11:10:21 2022] DELETE /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 19 11:10:22.024989 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-22607aa2-c86f-4e9e-8d54-e4474a4addaf tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:22.025681 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-22607aa2-c86f-4e9e-8d54-e4474a4addaf tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:22.031888 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-22607aa2-c86f-4e9e-8d54-e4474a4addaf tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:22.032323 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-22607aa2-c86f-4e9e-8d54-e4474a4addaf tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:22.033117 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-22607aa2-c86f-4e9e-8d54-e4474a4addaf tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:22.033556 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-22607aa2-c86f-4e9e-8d54-e4474a4addaf tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:22.159775 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-22607aa2-c86f-4e9e-8d54-e4474a4addaf tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:22.167229 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-22607aa2-c86f-4e9e-8d54-e4474a4addaf tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1507 microversion: 2.1 time: 0.147321 Jul 19 11:10:22.167812 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 460/897] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:22 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1507 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:23.185301 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-4e332ae3-b3b4-4a0f-86d7-fd0f9a149042 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:23.185910 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-4e332ae3-b3b4-4a0f-86d7-fd0f9a149042 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:23.192108 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4e332ae3-b3b4-4a0f-86d7-fd0f9a149042 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:23.192558 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4e332ae3-b3b4-4a0f-86d7-fd0f9a149042 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:23.193462 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4e332ae3-b3b4-4a0f-86d7-fd0f9a149042 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:23.193901 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4e332ae3-b3b4-4a0f-86d7-fd0f9a149042 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:23.324388 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-4e332ae3-b3b4-4a0f-86d7-fd0f9a149042 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:23.332379 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-4e332ae3-b3b4-4a0f-86d7-fd0f9a149042 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1507 microversion: 2.1 time: 0.151409 Jul 19 11:10:23.332980 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 438/898] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:23 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1507 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:24.349783 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-4c7f7a5e-2fbb-4040-8d5a-5085f603254d tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:24.350343 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-4c7f7a5e-2fbb-4040-8d5a-5085f603254d tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:24.357371 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4c7f7a5e-2fbb-4040-8d5a-5085f603254d tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:24.357823 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4c7f7a5e-2fbb-4040-8d5a-5085f603254d tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:24.358603 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4c7f7a5e-2fbb-4040-8d5a-5085f603254d tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:24.359043 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4c7f7a5e-2fbb-4040-8d5a-5085f603254d tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:24.489484 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-4c7f7a5e-2fbb-4040-8d5a-5085f603254d tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:24.497449 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-4c7f7a5e-2fbb-4040-8d5a-5085f603254d tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 200 len: 1507 microversion: 2.1 time: 0.151699 Jul 19 11:10:24.498110 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 461/899] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:24 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 1507 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:24.634389 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-74acc6e7-c58b-45f4-a055-c5b7666ab043 req-f21dbdc1-86d8-46e1-86bd-8c15feeccd01 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "49e783c8-8e6d-4789-b81d-8e1ca4a550a6", "name": "network-vif-deleted", "tag": "ba1dd015-6965-43ea-a7cf-059d7c358686"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:10:24.645194 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-74acc6e7-c58b-45f4-a055-c5b7666ab043 req-f21dbdc1-86d8-46e1-86bd-8c15feeccd01 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:24.645740 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-74acc6e7-c58b-45f4-a055-c5b7666ab043 req-f21dbdc1-86d8-46e1-86bd-8c15feeccd01 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:24.659065 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-74acc6e7-c58b-45f4-a055-c5b7666ab043 req-f21dbdc1-86d8-46e1-86bd-8c15feeccd01 service nova] Creating event network-vif-deleted:ba1dd015-6965-43ea-a7cf-059d7c358686 for instance 49e783c8-8e6d-4789-b81d-8e1ca4a550a6 on hv-cinder-maste Jul 19 11:10:24.664669 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-74acc6e7-c58b-45f4-a055-c5b7666ab043 req-f21dbdc1-86d8-46e1-86bd-8c15feeccd01 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034384 Jul 19 11:10:24.665847 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 439/900] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:10:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:10:25.515256 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-1667b630-0a87-4cd5-a873-fd6e78263352 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:25.515879 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-1667b630-0a87-4cd5-a873-fd6e78263352 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: 49e783c8-8e6d-4789-b81d-8e1ca4a550a6] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:25.522477 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1667b630-0a87-4cd5-a873-fd6e78263352 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:25.522917 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1667b630-0a87-4cd5-a873-fd6e78263352 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:25.523665 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1667b630-0a87-4cd5-a873-fd6e78263352 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:25.524097 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1667b630-0a87-4cd5-a873-fd6e78263352 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:25.537106 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.wsgi [None req-1667b630-0a87-4cd5-a873-fd6e78263352 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] HTTP exception thrown: Instance 49e783c8-8e6d-4789-b81d-8e1ca4a550a6 could not be found. Jul 19 11:10:25.538756 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-1667b630-0a87-4cd5-a873-fd6e78263352 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Request method failure captured: Jul 19 11:10:25.538756 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: request: GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 HTTP/1.1 Jul 19 11:10:25.538756 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept: application/json Jul 19 11:10:25.538756 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept-Encoding: identity Jul 19 11:10:25.538756 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Connection: close Jul 19 11:10:25.538756 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Length: 0 Jul 19 11:10:25.538756 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Type: application/json Jul 19 11:10:25.538756 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Host: 10.222.0.33 Jul 19 11:10:25.538756 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Openstack-System-Scope: None Jul 19 11:10:25.538756 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: User-Agent: python-urllib3/1.26.9 Jul 19 11:10:25.538756 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Auth-Token: gAAAAABi1o_D_huvBlD9ZJOIcq8Z1aOSnILmJp8MQgaQ8MuWsEAL_ZmZPsH--WyF20ahgvN08GveS7vk1bvyoVIPWYbHaxCp791BKfyC6yZ04rwPPH3vQlTPlGO54x_Ovlr-nuJTMqDQHYocrhpCc4nPGpje8DwiZIJCWR_YJdRwNKP8_4bwd5o Jul 19 11:10:25.538756 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Id: None Jul 19 11:10:25.538756 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Name: None Jul 19 11:10:25.538756 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Identity-Status: Confirmed Jul 19 11:10:25.538756 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Is-Admin-Project: True Jul 19 11:10:25.538756 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Id: default Jul 19 11:10:25.538756 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Name: Default Jul 19 11:10:25.538756 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Id: 05fceba2c70c40a399e71daabe8edef6 Jul 19 11:10:25.538756 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1654338802 Jul 19 11:10:25.538756 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Role: member,reader Jul 19 11:10:25.538756 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Roles: member,reader Jul 19 11:10:25.540392 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/05fceba2c70c40a399e71daabe8edef6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/05fceba2c70c40a399e71daabe8edef6"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/05fceba2c70c40a399e71daabe8edef6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_05fceba2c70c40a399e71daabe8edef6"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:10:25.540392 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1654338802 Jul 19 11:10:25.540392 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Id: 05fceba2c70c40a399e71daabe8edef6 Jul 19 11:10:25.540392 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1654338802 Jul 19 11:10:25.540392 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User: tempest-ServerRescueNegativeTestJSON-1654338802-project Jul 19 11:10:25.540392 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Id: default Jul 19 11:10:25.540392 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Name: Default Jul 19 11:10:25.540392 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Id: c95281b28eae450d8678583a929c3b8a Jul 19 11:10:25.540392 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1654338802-project Jul 19 11:10:25.540392 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: method: > Jul 19 11:10:25.540392 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: exception: Instance 49e783c8-8e6d-4789-b81d-8e1ca4a550a6 could not be found. Jul 19 11:10:25.540392 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: action_args: {'id': '49e783c8-8e6d-4789-b81d-8e1ca4a550a6'} Jul 19 11:10:25.541283 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:10:25.541283 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:25.541283 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:10:25.541283 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:10:25.541283 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:10:25.541283 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:10:25.541283 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:10:25.541283 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:10:25.541283 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:10:25.541283 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:10:25.541283 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 49e783c8-8e6d-4789-b81d-8e1ca4a550a6 could not be found. Jul 19 11:10:25.541283 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:25.541283 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:10:25.541283 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:25.541283 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:25.541283 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:10:25.541283 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:10:25.541283 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:10:25.541283 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:10:25.541283 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:10:25.542740 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:10:25.542740 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 19 11:10:25.542740 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 19 11:10:25.542740 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:10:25.542740 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:10:25.542740 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:10:25.542740 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:10:25.542740 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 49e783c8-8e6d-4789-b81d-8e1ca4a550a6 could not be found. Jul 19 11:10:25.542740 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:25.542740 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:10:25.542740 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:25.542740 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:25.542740 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:10:25.542740 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:10:25.542740 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:10:25.542740 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:10:25.542740 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 49e783c8-8e6d-4789-b81d-8e1ca4a550a6 could not be found. Jul 19 11:10:25.542740 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:25.544067 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-1667b630-0a87-4cd5-a873-fd6e78263352 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Returning 404 to user: Instance 49e783c8-8e6d-4789-b81d-8e1ca4a550a6 could not be found. {{(pid=117848) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:10:25.544067 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-1667b630-0a87-4cd5-a873-fd6e78263352 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6" status: 404 len: 111 microversion: 2.1 time: 0.029293 Jul 19 11:10:25.544067 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 462/901] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:25 2022] GET /compute/v2.1/servers/49e783c8-8e6d-4789-b81d-8e1ca4a550a6 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:10:25.550400 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-3a43ba88-45d1-4911-9ea0-e7c927396939 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:25.551235 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-3a43ba88-45d1-4911-9ea0-e7c927396939 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:25.559005 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3a43ba88-45d1-4911-9ea0-e7c927396939 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:25.559447 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3a43ba88-45d1-4911-9ea0-e7c927396939 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:25.560202 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3a43ba88-45d1-4911-9ea0-e7c927396939 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:25.560634 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3a43ba88-45d1-4911-9ea0-e7c927396939 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:25.586752 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-3a43ba88-45d1-4911-9ea0-e7c927396939 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Going to try to terminate instance {{(pid=117849) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 19 11:10:25.663625 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-3a43ba88-45d1-4911-9ea0-e7c927396939 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "DELETE /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 204 len: 0 microversion: 2.1 time: 0.118488 Jul 19 11:10:25.664581 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 440/902] 10.222.0.33 () {58 vars in 1238 bytes} [Tue Jul 19 11:10:25 2022] DELETE /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 19 11:10:25.673075 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-4bc94d46-f304-4530-acd6-8ffb882fc5f4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:25.673677 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-4bc94d46-f304-4530-acd6-8ffb882fc5f4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:25.679676 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4bc94d46-f304-4530-acd6-8ffb882fc5f4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:25.680126 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4bc94d46-f304-4530-acd6-8ffb882fc5f4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:25.680920 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4bc94d46-f304-4530-acd6-8ffb882fc5f4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:25.681360 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4bc94d46-f304-4530-acd6-8ffb882fc5f4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:25.813256 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-4bc94d46-f304-4530-acd6-8ffb882fc5f4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:25.820545 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-4bc94d46-f304-4530-acd6-8ffb882fc5f4 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1523 microversion: 2.1 time: 0.151832 Jul 19 11:10:25.821142 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 463/903] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:25 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1523 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:26.838441 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-aa7e5544-a1bb-4b76-aac9-ee57ac07ad9f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:26.839124 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-aa7e5544-a1bb-4b76-aac9-ee57ac07ad9f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:26.846098 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-aa7e5544-a1bb-4b76-aac9-ee57ac07ad9f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:26.846542 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-aa7e5544-a1bb-4b76-aac9-ee57ac07ad9f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:26.847317 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-aa7e5544-a1bb-4b76-aac9-ee57ac07ad9f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:26.847744 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-aa7e5544-a1bb-4b76-aac9-ee57ac07ad9f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:26.973754 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-aa7e5544-a1bb-4b76-aac9-ee57ac07ad9f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:26.981075 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-aa7e5544-a1bb-4b76-aac9-ee57ac07ad9f tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1523 microversion: 2.1 time: 0.146836 Jul 19 11:10:26.981630 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 441/904] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:26 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1523 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:27.999146 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-64b2eaf0-27c6-40af-8ebe-1aae643b7f07 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:28.000000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-64b2eaf0-27c6-40af-8ebe-1aae643b7f07 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:28.006028 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-64b2eaf0-27c6-40af-8ebe-1aae643b7f07 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:28.006473 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-64b2eaf0-27c6-40af-8ebe-1aae643b7f07 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:28.007233 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-64b2eaf0-27c6-40af-8ebe-1aae643b7f07 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:28.007655 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-64b2eaf0-27c6-40af-8ebe-1aae643b7f07 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:28.133084 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-64b2eaf0-27c6-40af-8ebe-1aae643b7f07 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:28.140507 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-64b2eaf0-27c6-40af-8ebe-1aae643b7f07 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1523 microversion: 2.1 time: 0.146023 Jul 19 11:10:28.141096 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 464/905] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:27 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1523 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:28.840017 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-1ac70d31-dfe0-4b33-a547-f4215320f36a req-3aa2b1ee-37ba-4149-a6b5-3f12b24123e9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c3ea5a95-376a-4482-b664-f069e803e258", "name": "network-vif-deleted", "tag": "1504f06f-d7c2-447b-acc1-5498633749e3"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:10:28.849950 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-1ac70d31-dfe0-4b33-a547-f4215320f36a req-3aa2b1ee-37ba-4149-a6b5-3f12b24123e9 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:28.850396 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-1ac70d31-dfe0-4b33-a547-f4215320f36a req-3aa2b1ee-37ba-4149-a6b5-3f12b24123e9 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:28.861575 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-1ac70d31-dfe0-4b33-a547-f4215320f36a req-3aa2b1ee-37ba-4149-a6b5-3f12b24123e9 service nova] Creating event network-vif-deleted:1504f06f-d7c2-447b-acc1-5498633749e3 for instance c3ea5a95-376a-4482-b664-f069e803e258 on hv-cinder-maste Jul 19 11:10:28.866273 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-1ac70d31-dfe0-4b33-a547-f4215320f36a req-3aa2b1ee-37ba-4149-a6b5-3f12b24123e9 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029948 Jul 19 11:10:28.867005 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 442/906] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:10:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:10:29.156454 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-46a2bf25-cb00-47fd-807a-048c47e57092 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:29.157096 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-46a2bf25-cb00-47fd-807a-048c47e57092 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:29.163624 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-46a2bf25-cb00-47fd-807a-048c47e57092 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:29.164066 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-46a2bf25-cb00-47fd-807a-048c47e57092 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:29.164895 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-46a2bf25-cb00-47fd-807a-048c47e57092 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:29.165327 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-46a2bf25-cb00-47fd-807a-048c47e57092 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:29.199610 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-46a2bf25-cb00-47fd-807a-048c47e57092 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:29.200049 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-46a2bf25-cb00-47fd-807a-048c47e57092 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:29.200496 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.objects.instance [None req-46a2bf25-cb00-47fd-807a-048c47e57092 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lazy-loading 'fault' on Instance uuid c3ea5a95-376a-4482-b664-f069e803e258 {{(pid=117848) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 19 11:10:29.246748 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-46a2bf25-cb00-47fd-807a-048c47e57092 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c95281b28eae450d8678583a929c3b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05fceba2c70c40a399e71daabe8edef6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:29.255306 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-46a2bf25-cb00-47fd-807a-048c47e57092 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 200 len: 1312 microversion: 2.1 time: 0.102792 Jul 19 11:10:29.255868 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 465/907] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:29 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 1312 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:30.273821 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-8b1db64c-7d63-43ea-adb8-d35fe28a5e23 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:30.274513 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-8b1db64c-7d63-43ea-adb8-d35fe28a5e23 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] [instance: c3ea5a95-376a-4482-b664-f069e803e258] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:30.281329 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8b1db64c-7d63-43ea-adb8-d35fe28a5e23 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:30.281779 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8b1db64c-7d63-43ea-adb8-d35fe28a5e23 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:30.282537 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8b1db64c-7d63-43ea-adb8-d35fe28a5e23 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:30.282975 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8b1db64c-7d63-43ea-adb8-d35fe28a5e23 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:30.295867 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.wsgi [None req-8b1db64c-7d63-43ea-adb8-d35fe28a5e23 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] HTTP exception thrown: Instance c3ea5a95-376a-4482-b664-f069e803e258 could not be found. Jul 19 11:10:30.297471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-8b1db64c-7d63-43ea-adb8-d35fe28a5e23 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Request method failure captured: Jul 19 11:10:30.297471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: request: GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 HTTP/1.1 Jul 19 11:10:30.297471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept: application/json Jul 19 11:10:30.297471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept-Encoding: identity Jul 19 11:10:30.297471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Connection: close Jul 19 11:10:30.297471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Length: 0 Jul 19 11:10:30.297471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Type: application/json Jul 19 11:10:30.297471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Host: 10.222.0.33 Jul 19 11:10:30.297471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Openstack-System-Scope: None Jul 19 11:10:30.297471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: User-Agent: python-urllib3/1.26.9 Jul 19 11:10:30.297471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Auth-Token: gAAAAABi1o_D_huvBlD9ZJOIcq8Z1aOSnILmJp8MQgaQ8MuWsEAL_ZmZPsH--WyF20ahgvN08GveS7vk1bvyoVIPWYbHaxCp791BKfyC6yZ04rwPPH3vQlTPlGO54x_Ovlr-nuJTMqDQHYocrhpCc4nPGpje8DwiZIJCWR_YJdRwNKP8_4bwd5o Jul 19 11:10:30.297471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Id: None Jul 19 11:10:30.297471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Name: None Jul 19 11:10:30.297471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Identity-Status: Confirmed Jul 19 11:10:30.297471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Is-Admin-Project: True Jul 19 11:10:30.297471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Id: default Jul 19 11:10:30.297471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Name: Default Jul 19 11:10:30.297471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Id: 05fceba2c70c40a399e71daabe8edef6 Jul 19 11:10:30.297471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1654338802 Jul 19 11:10:30.297471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Role: member,reader Jul 19 11:10:30.297471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Roles: member,reader Jul 19 11:10:30.298991 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/05fceba2c70c40a399e71daabe8edef6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/05fceba2c70c40a399e71daabe8edef6"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/05fceba2c70c40a399e71daabe8edef6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_05fceba2c70c40a399e71daabe8edef6"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:10:30.298991 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1654338802 Jul 19 11:10:30.298991 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Id: 05fceba2c70c40a399e71daabe8edef6 Jul 19 11:10:30.298991 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1654338802 Jul 19 11:10:30.298991 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User: tempest-ServerRescueNegativeTestJSON-1654338802-project Jul 19 11:10:30.298991 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Id: default Jul 19 11:10:30.298991 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Name: Default Jul 19 11:10:30.298991 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Id: c95281b28eae450d8678583a929c3b8a Jul 19 11:10:30.298991 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1654338802-project Jul 19 11:10:30.298991 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: method: > Jul 19 11:10:30.298991 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: exception: Instance c3ea5a95-376a-4482-b664-f069e803e258 could not be found. Jul 19 11:10:30.298991 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: action_args: {'id': 'c3ea5a95-376a-4482-b664-f069e803e258'} Jul 19 11:10:30.299858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:10:30.299858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:30.299858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:10:30.299858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:10:30.299858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:10:30.299858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:10:30.299858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:10:30.299858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:10:30.299858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:10:30.299858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:10:30.299858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c3ea5a95-376a-4482-b664-f069e803e258 could not be found. Jul 19 11:10:30.299858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:30.299858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:10:30.299858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:30.299858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:30.299858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:10:30.299858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:10:30.299858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:10:30.299858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:10:30.299858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:10:30.301407 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:10:30.301407 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 19 11:10:30.301407 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 19 11:10:30.301407 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:10:30.301407 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:10:30.301407 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:10:30.301407 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:10:30.301407 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c3ea5a95-376a-4482-b664-f069e803e258 could not be found. Jul 19 11:10:30.301407 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:30.301407 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:10:30.301407 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:30.301407 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:30.301407 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:10:30.301407 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:10:30.301407 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:10:30.301407 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:10:30.301407 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c3ea5a95-376a-4482-b664-f069e803e258 could not be found. Jul 19 11:10:30.301407 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:30.302674 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-8b1db64c-7d63-43ea-adb8-d35fe28a5e23 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] Returning 404 to user: Instance c3ea5a95-376a-4482-b664-f069e803e258 could not be found. {{(pid=117849) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:10:30.302674 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-8b1db64c-7d63-43ea-adb8-d35fe28a5e23 tempest-ServerRescueNegativeTestJSON-1654338802 tempest-ServerRescueNegativeTestJSON-1654338802-project] 10.222.0.33 "GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258" status: 404 len: 111 microversion: 2.1 time: 0.029705 Jul 19 11:10:30.302674 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 443/908] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:30 2022] GET /compute/v2.1/servers/c3ea5a95-376a-4482-b664-f069e803e258 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:10:32.953199 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-74405d8b-648d-4ce7-8713-9a233c6cd305 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:32.954388 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-74405d8b-648d-4ce7-8713-9a233c6cd305 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:32.960806 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-74405d8b-648d-4ce7-8713-9a233c6cd305 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:32.961255 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-74405d8b-648d-4ce7-8713-9a233c6cd305 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:32.962015 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-74405d8b-648d-4ce7-8713-9a233c6cd305 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:32.962445 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-74405d8b-648d-4ce7-8713-9a233c6cd305 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:32.985818 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-74405d8b-648d-4ce7-8713-9a233c6cd305 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9/ips" status: 200 len: 151 microversion: 2.20 time: 0.037156 Jul 19 11:10:32.986405 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 466/909] 10.222.0.33 () {60 vars in 1288 bytes} [Tue Jul 19 11:10:32 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9/ips => generated 151 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 19 11:10:34.631680 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-91fe9308-5267-4a3e-8375-89a130c392e0 tempest-VolumesNegativeTest-790337672 tempest-VolumesNegativeTest-790337672-project] Action: 'create', calling method: .version_select at 0x7f04aa2aa4c0>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1260978278", "metadata": {"Type": "work"}}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:10:34.650264 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-91fe9308-5267-4a3e-8375-89a130c392e0 tempest-VolumesNegativeTest-790337672 tempest-VolumesNegativeTest-790337672-project] Request method failure captured: Jul 19 11:10:34.650264 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Jul 19 11:10:34.650264 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept: application/json Jul 19 11:10:34.650264 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept-Encoding: identity Jul 19 11:10:34.650264 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Connection: close Jul 19 11:10:34.650264 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Length: 122 Jul 19 11:10:34.650264 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Type: application/json Jul 19 11:10:34.650264 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Host: 10.222.0.33 Jul 19 11:10:34.650264 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Openstack-System-Scope: None Jul 19 11:10:34.650264 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: User-Agent: python-urllib3/1.26.9 Jul 19 11:10:34.650264 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Auth-Token: gAAAAABi1pEqJQH-2K-crnGbryAI8N_9iTtzsh_CJ_MXIFbrtpdGD8DfmTWxfnwy-deR4MrZo4Cm_RTgIQURemxGkOMXk9BPJ5kl2lgxPBRvldajb5NgAs-bA-3t_g2HJboxY0mOF2NX6cX0YWZl923qjv1RKXiOQuhlUE_KIWxDtPSJCaImagg Jul 19 11:10:34.650264 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Id: None Jul 19 11:10:34.650264 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Name: None Jul 19 11:10:34.650264 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Identity-Status: Confirmed Jul 19 11:10:34.650264 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Is-Admin-Project: True Jul 19 11:10:34.650264 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Id: default Jul 19 11:10:34.650264 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Name: Default Jul 19 11:10:34.650264 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Id: d916fb4d05694bd5ba3f9d61678fba03 Jul 19 11:10:34.650264 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Name: tempest-VolumesNegativeTest-790337672 Jul 19 11:10:34.650264 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Role: member,reader Jul 19 11:10:34.650264 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Roles: member,reader Jul 19 11:10:34.651828 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/d916fb4d05694bd5ba3f9d61678fba03"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/d916fb4d05694bd5ba3f9d61678fba03"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/d916fb4d05694bd5ba3f9d61678fba03"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_d916fb4d05694bd5ba3f9d61678fba03"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:10:34.651828 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant: tempest-VolumesNegativeTest-790337672 Jul 19 11:10:34.651828 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Id: d916fb4d05694bd5ba3f9d61678fba03 Jul 19 11:10:34.651828 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Name: tempest-VolumesNegativeTest-790337672 Jul 19 11:10:34.651828 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User: tempest-VolumesNegativeTest-790337672-project Jul 19 11:10:34.651828 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Id: default Jul 19 11:10:34.651828 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Name: Default Jul 19 11:10:34.651828 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Id: 5f95631bc447497ba062a57c877a133b Jul 19 11:10:34.651828 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Name: tempest-VolumesNegativeTest-790337672-project Jul 19 11:10:34.651828 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Jul 19 11:10:34.651828 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1260978278", "metadata": {"Type": "work"}}} Jul 19 11:10:34.651828 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: method: Jul 19 11:10:34.651828 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Jul 19 11:10:34.652809 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-1260978278', 'metadata': {'Type': 'work'}}}} Jul 19 11:10:34.652809 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:10:34.652809 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:34.652809 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 19 11:10:34.652809 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 19 11:10:34.652809 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 19 11:10:34.652809 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise error Jul 19 11:10:34.652809 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Jul 19 11:10:34.652809 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.652809 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Jul 19 11:10:34.652809 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Jul 19 11:10:34.652809 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.652809 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi On instance['volume']: Jul 19 11:10:34.652809 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi '#$%' Jul 19 11:10:34.652809 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.652809 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:10:34.652809 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.652809 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:34.652809 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:10:34.652809 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:10:34.652809 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:10:34.652809 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:10:34.654375 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 19 11:10:34.654375 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 19 11:10:34.654375 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:10:34.654375 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:10:34.654375 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 19 11:10:34.654375 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 19 11:10:34.654375 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 19 11:10:34.654375 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 19 11:10:34.654375 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 19 11:10:34.654375 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 19 11:10:34.654375 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Jul 19 11:10:34.654375 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.654375 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:10:34.654375 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.654375 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:34.654375 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:10:34.654375 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:10:34.654375 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 19 11:10:34.654375 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 19 11:10:34.656361 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Jul 19 11:10:34.656361 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.656361 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-91fe9308-5267-4a3e-8375-89a130c392e0 tempest-VolumesNegativeTest-790337672 tempest-VolumesNegativeTest-790337672-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=117849) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:10:34.656361 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-91fe9308-5267-4a3e-8375-89a130c392e0 tempest-VolumesNegativeTest-790337672 tempest-VolumesNegativeTest-790337672-project] 10.222.0.33 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.139009 Jul 19 11:10:34.656361 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 444/910] 10.222.0.33 () {60 vars in 1155 bytes} [Tue Jul 19 11:10:34 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 140 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 19 11:10:34.667482 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-1f42e046-65cd-4c03-a88c-6994de4a5646 tempest-VolumesNegativeTest-790337672 tempest-VolumesNegativeTest-790337672-project] Action: 'create', calling method: .version_select at 0x7f04aa4d8700>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-2032618426", "metadata": {"Type": "work"}}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:10:34.670771 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-1f42e046-65cd-4c03-a88c-6994de4a5646 tempest-VolumesNegativeTest-790337672 tempest-VolumesNegativeTest-790337672-project] Request method failure captured: Jul 19 11:10:34.670771 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Jul 19 11:10:34.670771 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept: application/json Jul 19 11:10:34.670771 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept-Encoding: identity Jul 19 11:10:34.670771 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Connection: close Jul 19 11:10:34.670771 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Length: 120 Jul 19 11:10:34.670771 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Type: application/json Jul 19 11:10:34.670771 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Host: 10.222.0.33 Jul 19 11:10:34.670771 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Openstack-System-Scope: None Jul 19 11:10:34.670771 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: User-Agent: python-urllib3/1.26.9 Jul 19 11:10:34.670771 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Auth-Token: gAAAAABi1pEqJQH-2K-crnGbryAI8N_9iTtzsh_CJ_MXIFbrtpdGD8DfmTWxfnwy-deR4MrZo4Cm_RTgIQURemxGkOMXk9BPJ5kl2lgxPBRvldajb5NgAs-bA-3t_g2HJboxY0mOF2NX6cX0YWZl923qjv1RKXiOQuhlUE_KIWxDtPSJCaImagg Jul 19 11:10:34.670771 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Id: None Jul 19 11:10:34.670771 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Name: None Jul 19 11:10:34.670771 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Identity-Status: Confirmed Jul 19 11:10:34.670771 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Is-Admin-Project: True Jul 19 11:10:34.670771 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Id: default Jul 19 11:10:34.670771 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Name: Default Jul 19 11:10:34.670771 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Id: d916fb4d05694bd5ba3f9d61678fba03 Jul 19 11:10:34.670771 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Name: tempest-VolumesNegativeTest-790337672 Jul 19 11:10:34.670771 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Role: member,reader Jul 19 11:10:34.670771 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Roles: member,reader Jul 19 11:10:34.672319 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/d916fb4d05694bd5ba3f9d61678fba03"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/d916fb4d05694bd5ba3f9d61678fba03"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/d916fb4d05694bd5ba3f9d61678fba03"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_d916fb4d05694bd5ba3f9d61678fba03"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:10:34.672319 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant: tempest-VolumesNegativeTest-790337672 Jul 19 11:10:34.672319 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Id: d916fb4d05694bd5ba3f9d61678fba03 Jul 19 11:10:34.672319 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Name: tempest-VolumesNegativeTest-790337672 Jul 19 11:10:34.672319 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User: tempest-VolumesNegativeTest-790337672-project Jul 19 11:10:34.672319 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Id: default Jul 19 11:10:34.672319 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Name: Default Jul 19 11:10:34.672319 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Id: 5f95631bc447497ba062a57c877a133b Jul 19 11:10:34.672319 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Name: tempest-VolumesNegativeTest-790337672-project Jul 19 11:10:34.672319 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Jul 19 11:10:34.672319 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-2032618426", "metadata": {"Type": "work"}}} Jul 19 11:10:34.672319 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: method: Jul 19 11:10:34.672319 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Jul 19 11:10:34.673297 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-2032618426', 'metadata': {'Type': 'work'}}}} Jul 19 11:10:34.673297 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:10:34.673297 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:34.673297 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 19 11:10:34.673297 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 19 11:10:34.673297 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 19 11:10:34.673297 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise error Jul 19 11:10:34.673297 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Jul 19 11:10:34.673297 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.673297 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Jul 19 11:10:34.673297 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Jul 19 11:10:34.673297 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.673297 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi On instance['volume']: Jul 19 11:10:34.673297 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi '0' Jul 19 11:10:34.673297 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.673297 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:10:34.673297 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.673297 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:34.673297 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:10:34.673297 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:10:34.673297 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:10:34.673297 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:10:34.674828 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 19 11:10:34.674828 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 19 11:10:34.674828 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:10:34.674828 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:10:34.674828 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 19 11:10:34.674828 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 19 11:10:34.674828 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 19 11:10:34.674828 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 19 11:10:34.674828 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 19 11:10:34.674828 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 19 11:10:34.674828 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Jul 19 11:10:34.674828 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.674828 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:10:34.674828 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.674828 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:34.674828 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:10:34.674828 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:10:34.674828 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 19 11:10:34.674828 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 19 11:10:34.676045 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Jul 19 11:10:34.676045 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.676045 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-1f42e046-65cd-4c03-a88c-6994de4a5646 tempest-VolumesNegativeTest-790337672 tempest-VolumesNegativeTest-790337672-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=117848) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:10:34.676045 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-1f42e046-65cd-4c03-a88c-6994de4a5646 tempest-VolumesNegativeTest-790337672 tempest-VolumesNegativeTest-790337672-project] 10.222.0.33 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.008637 Jul 19 11:10:34.676045 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 467/911] 10.222.0.33 () {60 vars in 1155 bytes} [Tue Jul 19 11:10:34 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 15 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 19 11:10:34.682302 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-bca45ad0-1c14-4b86-9d3e-219cf0b2fa75 tempest-VolumesNegativeTest-790337672 tempest-VolumesNegativeTest-790337672-project] Action: 'create', calling method: .version_select at 0x7f04aa1b23a0>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-608897639", "metadata": {"Type": "work"}}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:10:34.685380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-bca45ad0-1c14-4b86-9d3e-219cf0b2fa75 tempest-VolumesNegativeTest-790337672 tempest-VolumesNegativeTest-790337672-project] Request method failure captured: Jul 19 11:10:34.685380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Jul 19 11:10:34.685380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept: application/json Jul 19 11:10:34.685380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept-Encoding: identity Jul 19 11:10:34.685380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Connection: close Jul 19 11:10:34.685380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Length: 118 Jul 19 11:10:34.685380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Type: application/json Jul 19 11:10:34.685380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Host: 10.222.0.33 Jul 19 11:10:34.685380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Openstack-System-Scope: None Jul 19 11:10:34.685380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: User-Agent: python-urllib3/1.26.9 Jul 19 11:10:34.685380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Auth-Token: gAAAAABi1pEqJQH-2K-crnGbryAI8N_9iTtzsh_CJ_MXIFbrtpdGD8DfmTWxfnwy-deR4MrZo4Cm_RTgIQURemxGkOMXk9BPJ5kl2lgxPBRvldajb5NgAs-bA-3t_g2HJboxY0mOF2NX6cX0YWZl923qjv1RKXiOQuhlUE_KIWxDtPSJCaImagg Jul 19 11:10:34.685380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Id: None Jul 19 11:10:34.685380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Name: None Jul 19 11:10:34.685380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Identity-Status: Confirmed Jul 19 11:10:34.685380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Is-Admin-Project: True Jul 19 11:10:34.685380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Id: default Jul 19 11:10:34.685380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Name: Default Jul 19 11:10:34.685380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Id: d916fb4d05694bd5ba3f9d61678fba03 Jul 19 11:10:34.685380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Name: tempest-VolumesNegativeTest-790337672 Jul 19 11:10:34.685380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Role: member,reader Jul 19 11:10:34.685380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Roles: member,reader Jul 19 11:10:34.686852 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/d916fb4d05694bd5ba3f9d61678fba03"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/d916fb4d05694bd5ba3f9d61678fba03"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/d916fb4d05694bd5ba3f9d61678fba03"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_d916fb4d05694bd5ba3f9d61678fba03"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:10:34.686852 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant: tempest-VolumesNegativeTest-790337672 Jul 19 11:10:34.686852 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Id: d916fb4d05694bd5ba3f9d61678fba03 Jul 19 11:10:34.686852 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Name: tempest-VolumesNegativeTest-790337672 Jul 19 11:10:34.686852 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User: tempest-VolumesNegativeTest-790337672-project Jul 19 11:10:34.686852 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Id: default Jul 19 11:10:34.686852 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Name: Default Jul 19 11:10:34.686852 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Id: 5f95631bc447497ba062a57c877a133b Jul 19 11:10:34.686852 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Name: tempest-VolumesNegativeTest-790337672-project Jul 19 11:10:34.686852 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Jul 19 11:10:34.686852 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-608897639", "metadata": {"Type": "work"}}} Jul 19 11:10:34.686852 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: method: Jul 19 11:10:34.686852 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Jul 19 11:10:34.687838 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-608897639', 'metadata': {'Type': 'work'}}}} Jul 19 11:10:34.687838 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:10:34.687838 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:34.687838 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 19 11:10:34.687838 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 19 11:10:34.687838 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 19 11:10:34.687838 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise error Jul 19 11:10:34.687838 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Jul 19 11:10:34.687838 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.687838 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Jul 19 11:10:34.687838 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Jul 19 11:10:34.687838 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.687838 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi On instance['volume']: Jul 19 11:10:34.687838 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi '' Jul 19 11:10:34.687838 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.687838 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:10:34.687838 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.687838 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:34.687838 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:10:34.687838 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:10:34.687838 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:10:34.687838 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:10:34.689422 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 19 11:10:34.689422 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 19 11:10:34.689422 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:10:34.689422 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:10:34.689422 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 19 11:10:34.689422 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 19 11:10:34.689422 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 19 11:10:34.689422 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 19 11:10:34.689422 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 19 11:10:34.689422 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 19 11:10:34.689422 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Jul 19 11:10:34.689422 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.689422 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:10:34.689422 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.689422 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:34.689422 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:10:34.689422 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:10:34.689422 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 19 11:10:34.689422 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 19 11:10:34.690778 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Jul 19 11:10:34.690778 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.690778 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-bca45ad0-1c14-4b86-9d3e-219cf0b2fa75 tempest-VolumesNegativeTest-790337672 tempest-VolumesNegativeTest-790337672-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=117849) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:10:34.690778 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-bca45ad0-1c14-4b86-9d3e-219cf0b2fa75 tempest-VolumesNegativeTest-790337672 tempest-VolumesNegativeTest-790337672-project] 10.222.0.33 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.008497 Jul 19 11:10:34.690778 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 445/912] 10.222.0.33 () {60 vars in 1155 bytes} [Tue Jul 19 11:10:34 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 9 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 19 11:10:34.696359 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-2652ae75-fa22-4164-8679-8b24874bda7e tempest-VolumesNegativeTest-790337672 tempest-VolumesNegativeTest-790337672-project] Calling method '.version_select at 0x7f04aa1f1af0>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:34.725633 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.wsgi [None req-2652ae75-fa22-4164-8679-8b24874bda7e tempest-VolumesNegativeTest-790337672 tempest-VolumesNegativeTest-790337672-project] HTTP exception thrown: Volume tempest-invalid-539602816 could not be found. Jul 19 11:10:34.727569 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-2652ae75-fa22-4164-8679-8b24874bda7e tempest-VolumesNegativeTest-790337672 tempest-VolumesNegativeTest-790337672-project] Request method failure captured: Jul 19 11:10:34.727569 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-539602816 HTTP/1.1 Jul 19 11:10:34.727569 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept: application/json Jul 19 11:10:34.727569 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept-Encoding: identity Jul 19 11:10:34.727569 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Connection: close Jul 19 11:10:34.727569 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Length: 0 Jul 19 11:10:34.727569 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Type: application/json Jul 19 11:10:34.727569 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Host: 10.222.0.33 Jul 19 11:10:34.727569 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Openstack-System-Scope: None Jul 19 11:10:34.727569 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: User-Agent: python-urllib3/1.26.9 Jul 19 11:10:34.727569 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Auth-Token: gAAAAABi1pEqJQH-2K-crnGbryAI8N_9iTtzsh_CJ_MXIFbrtpdGD8DfmTWxfnwy-deR4MrZo4Cm_RTgIQURemxGkOMXk9BPJ5kl2lgxPBRvldajb5NgAs-bA-3t_g2HJboxY0mOF2NX6cX0YWZl923qjv1RKXiOQuhlUE_KIWxDtPSJCaImagg Jul 19 11:10:34.727569 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Id: None Jul 19 11:10:34.727569 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Name: None Jul 19 11:10:34.727569 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Identity-Status: Confirmed Jul 19 11:10:34.727569 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Is-Admin-Project: True Jul 19 11:10:34.727569 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Id: default Jul 19 11:10:34.727569 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Name: Default Jul 19 11:10:34.727569 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Id: d916fb4d05694bd5ba3f9d61678fba03 Jul 19 11:10:34.727569 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Name: tempest-VolumesNegativeTest-790337672 Jul 19 11:10:34.727569 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Role: member,reader Jul 19 11:10:34.727569 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Roles: member,reader Jul 19 11:10:34.729136 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/d916fb4d05694bd5ba3f9d61678fba03"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/d916fb4d05694bd5ba3f9d61678fba03"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/d916fb4d05694bd5ba3f9d61678fba03"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_d916fb4d05694bd5ba3f9d61678fba03"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:10:34.729136 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant: tempest-VolumesNegativeTest-790337672 Jul 19 11:10:34.729136 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Id: d916fb4d05694bd5ba3f9d61678fba03 Jul 19 11:10:34.729136 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Name: tempest-VolumesNegativeTest-790337672 Jul 19 11:10:34.729136 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User: tempest-VolumesNegativeTest-790337672-project Jul 19 11:10:34.729136 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Id: default Jul 19 11:10:34.729136 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Name: Default Jul 19 11:10:34.729136 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Id: 5f95631bc447497ba062a57c877a133b Jul 19 11:10:34.729136 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Name: tempest-VolumesNegativeTest-790337672-project Jul 19 11:10:34.729136 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: method: Jul 19 11:10:34.729136 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: exception: Volume tempest-invalid-539602816 could not be found. Jul 19 11:10:34.729136 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: action_args: {'id': 'tempest-invalid-539602816'} Jul 19 11:10:34.729136 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:10:34.730086 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:34.730086 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 19 11:10:34.730086 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 19 11:10:34.730086 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 19 11:10:34.730086 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 19 11:10:34.730086 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 19 11:10:34.730086 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 19 11:10:34.730086 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 19 11:10:34.730086 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 19 11:10:34.730086 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 19 11:10:34.730086 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 19 11:10:34.730086 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 19 11:10:34.730086 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 19 11:10:34.730086 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 19 11:10:34.730086 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 19 11:10:34.730086 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-539602816 could not be found. (HTTP 404) (Request-ID: req-27a0f74c-d9ad-4c40-b625-bbea9a9b02fb) Jul 19 11:10:34.730086 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.730086 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:10:34.730086 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.731522 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:34.731522 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Jul 19 11:10:34.731522 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Jul 19 11:10:34.731522 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 19 11:10:34.731522 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 19 11:10:34.731522 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 19 11:10:34.731522 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 19 11:10:34.731522 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 19 11:10:34.731522 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 19 11:10:34.731522 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 19 11:10:34.731522 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 19 11:10:34.731522 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 19 11:10:34.731522 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 19 11:10:34.731522 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 19 11:10:34.731522 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 19 11:10:34.731522 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 19 11:10:34.731522 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 19 11:10:34.731522 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 19 11:10:34.731522 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 19 11:10:34.732888 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 19 11:10:34.732888 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 19 11:10:34.732888 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 19 11:10:34.732888 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 19 11:10:34.732888 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-539602816 could not be found. Jul 19 11:10:34.732888 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.732888 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:10:34.732888 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.732888 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:34.732888 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:10:34.732888 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:10:34.732888 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:10:34.732888 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:10:34.732888 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 19 11:10:34.732888 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 19 11:10:34.732888 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:10:34.732888 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:10:34.732888 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Jul 19 11:10:34.732888 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:10:34.732888 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-539602816 could not be found. Jul 19 11:10:34.734404 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.734404 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:10:34.734404 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.734404 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:34.734404 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:10:34.734404 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:10:34.734404 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:10:34.734404 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:10:34.734404 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-539602816 could not be found. Jul 19 11:10:34.734404 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.734404 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-2652ae75-fa22-4164-8679-8b24874bda7e tempest-VolumesNegativeTest-790337672 tempest-VolumesNegativeTest-790337672-project] Returning 404 to user: Volume tempest-invalid-539602816 could not be found. {{(pid=117848) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:10:34.734404 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-2652ae75-fa22-4164-8679-8b24874bda7e tempest-VolumesNegativeTest-790337672 tempest-VolumesNegativeTest-790337672-project] 10.222.0.33 "DELETE /compute/v2.1/os-volumes/tempest-invalid-539602816" status: 404 len: 98 microversion: 2.1 time: 0.036819 Jul 19 11:10:34.734404 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 468/913] 10.222.0.33 () {58 vars in 1214 bytes} [Tue Jul 19 11:10:34 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-539602816 => generated 98 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 19 11:10:34.738683 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-9bd09776-dc45-42c3-ac7e-398845a37fad tempest-VolumesNegativeTest-790337672 tempest-VolumesNegativeTest-790337672-project] 10.222.0.33 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.003760 Jul 19 11:10:34.739179 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 446/914] 10.222.0.33 () {58 vars in 1139 bytes} [Tue Jul 19 11:10:34 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 5 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Jul 19 11:10:34.746938 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-15d5198a-03a7-4e9b-9c18-c0b369a13ebf tempest-VolumesNegativeTest-790337672 tempest-VolumesNegativeTest-790337672-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.003554 Jul 19 11:10:34.747527 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 469/915] 10.222.0.33 () {58 vars in 1136 bytes} [Tue Jul 19 11:10:34 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 5 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Jul 19 11:10:34.756155 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-bc8b19ad-2085-4562-b6c0-102ba89a3148 tempest-VolumesNegativeTest-790337672 tempest-VolumesNegativeTest-790337672-project] Calling method '.version_select at 0x7f04aa2aa430>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:34.783767 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.wsgi [None req-bc8b19ad-2085-4562-b6c0-102ba89a3148 tempest-VolumesNegativeTest-790337672 tempest-VolumesNegativeTest-790337672-project] HTTP exception thrown: Volume 7d217148-ac19-4ef7-9837-51ae05204ac8 could not be found. Jul 19 11:10:34.785752 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-bc8b19ad-2085-4562-b6c0-102ba89a3148 tempest-VolumesNegativeTest-790337672 tempest-VolumesNegativeTest-790337672-project] Request method failure captured: Jul 19 11:10:34.785752 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: request: DELETE /compute/v2.1/os-volumes/7d217148-ac19-4ef7-9837-51ae05204ac8 HTTP/1.1 Jul 19 11:10:34.785752 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept: application/json Jul 19 11:10:34.785752 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept-Encoding: identity Jul 19 11:10:34.785752 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Connection: close Jul 19 11:10:34.785752 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Length: 0 Jul 19 11:10:34.785752 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Type: application/json Jul 19 11:10:34.785752 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Host: 10.222.0.33 Jul 19 11:10:34.785752 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Openstack-System-Scope: None Jul 19 11:10:34.785752 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: User-Agent: python-urllib3/1.26.9 Jul 19 11:10:34.785752 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Auth-Token: gAAAAABi1pEqJQH-2K-crnGbryAI8N_9iTtzsh_CJ_MXIFbrtpdGD8DfmTWxfnwy-deR4MrZo4Cm_RTgIQURemxGkOMXk9BPJ5kl2lgxPBRvldajb5NgAs-bA-3t_g2HJboxY0mOF2NX6cX0YWZl923qjv1RKXiOQuhlUE_KIWxDtPSJCaImagg Jul 19 11:10:34.785752 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Id: None Jul 19 11:10:34.785752 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Name: None Jul 19 11:10:34.785752 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Identity-Status: Confirmed Jul 19 11:10:34.785752 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Is-Admin-Project: True Jul 19 11:10:34.785752 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Id: default Jul 19 11:10:34.785752 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Name: Default Jul 19 11:10:34.785752 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Id: d916fb4d05694bd5ba3f9d61678fba03 Jul 19 11:10:34.785752 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Name: tempest-VolumesNegativeTest-790337672 Jul 19 11:10:34.785752 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Role: member,reader Jul 19 11:10:34.785752 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Roles: member,reader Jul 19 11:10:34.787250 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/d916fb4d05694bd5ba3f9d61678fba03"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/d916fb4d05694bd5ba3f9d61678fba03"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/d916fb4d05694bd5ba3f9d61678fba03"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_d916fb4d05694bd5ba3f9d61678fba03"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:10:34.787250 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant: tempest-VolumesNegativeTest-790337672 Jul 19 11:10:34.787250 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Id: d916fb4d05694bd5ba3f9d61678fba03 Jul 19 11:10:34.787250 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Name: tempest-VolumesNegativeTest-790337672 Jul 19 11:10:34.787250 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User: tempest-VolumesNegativeTest-790337672-project Jul 19 11:10:34.787250 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Id: default Jul 19 11:10:34.787250 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Name: Default Jul 19 11:10:34.787250 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Id: 5f95631bc447497ba062a57c877a133b Jul 19 11:10:34.787250 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Name: tempest-VolumesNegativeTest-790337672-project Jul 19 11:10:34.787250 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: method: Jul 19 11:10:34.787250 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: exception: Volume 7d217148-ac19-4ef7-9837-51ae05204ac8 could not be found. Jul 19 11:10:34.787250 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: action_args: {'id': '7d217148-ac19-4ef7-9837-51ae05204ac8'} Jul 19 11:10:34.787250 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:10:34.788165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:34.788165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 19 11:10:34.788165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 19 11:10:34.788165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 19 11:10:34.788165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 19 11:10:34.788165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 19 11:10:34.788165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 19 11:10:34.788165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 19 11:10:34.788165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 19 11:10:34.788165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 19 11:10:34.788165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 19 11:10:34.788165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 19 11:10:34.788165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 19 11:10:34.788165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 19 11:10:34.788165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 19 11:10:34.788165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 7d217148-ac19-4ef7-9837-51ae05204ac8 could not be found. (HTTP 404) (Request-ID: req-4b6ea97a-ffa7-443f-87d6-c064ee3b776d) Jul 19 11:10:34.788165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.788165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:10:34.788165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.789484 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:34.789484 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Jul 19 11:10:34.789484 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Jul 19 11:10:34.789484 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 19 11:10:34.789484 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 19 11:10:34.789484 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 19 11:10:34.789484 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 19 11:10:34.789484 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 19 11:10:34.789484 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 19 11:10:34.789484 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 19 11:10:34.789484 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 19 11:10:34.789484 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 19 11:10:34.789484 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 19 11:10:34.789484 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 19 11:10:34.789484 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 19 11:10:34.789484 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 19 11:10:34.789484 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 19 11:10:34.789484 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 19 11:10:34.789484 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 19 11:10:34.790717 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 19 11:10:34.790717 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 19 11:10:34.790717 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 19 11:10:34.790717 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 19 11:10:34.790717 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 7d217148-ac19-4ef7-9837-51ae05204ac8 could not be found. Jul 19 11:10:34.790717 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.790717 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:10:34.790717 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.790717 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:34.790717 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:10:34.790717 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:10:34.790717 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:10:34.790717 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:10:34.790717 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 19 11:10:34.790717 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 19 11:10:34.790717 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:10:34.790717 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:10:34.790717 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Jul 19 11:10:34.790717 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:10:34.791831 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 7d217148-ac19-4ef7-9837-51ae05204ac8 could not be found. Jul 19 11:10:34.791831 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.791831 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:10:34.791831 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.791831 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:34.791831 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:10:34.791831 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:10:34.791831 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:10:34.791831 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:10:34.791831 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 7d217148-ac19-4ef7-9837-51ae05204ac8 could not be found. Jul 19 11:10:34.791831 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.791831 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-bc8b19ad-2085-4562-b6c0-102ba89a3148 tempest-VolumesNegativeTest-790337672 tempest-VolumesNegativeTest-790337672-project] Returning 404 to user: Volume 7d217148-ac19-4ef7-9837-51ae05204ac8 could not be found. {{(pid=117849) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:10:34.791831 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-bc8b19ad-2085-4562-b6c0-102ba89a3148 tempest-VolumesNegativeTest-790337672 tempest-VolumesNegativeTest-790337672-project] 10.222.0.33 "DELETE /compute/v2.1/os-volumes/7d217148-ac19-4ef7-9837-51ae05204ac8" status: 404 len: 109 microversion: 2.1 time: 0.034429 Jul 19 11:10:34.791831 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 447/916] 10.222.0.33 () {58 vars in 1247 bytes} [Tue Jul 19 11:10:34 2022] DELETE /compute/v2.1/os-volumes/7d217148-ac19-4ef7-9837-51ae05204ac8 => generated 109 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 19 11:10:34.797382 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-e721150d-8d0d-45e2-a575-e605897071b5 tempest-VolumesNegativeTest-790337672 tempest-VolumesNegativeTest-790337672-project] Calling method '.version_select at 0x7f04aa4d8940>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:34.824532 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.wsgi [None req-e721150d-8d0d-45e2-a575-e605897071b5 tempest-VolumesNegativeTest-790337672 tempest-VolumesNegativeTest-790337672-project] HTTP exception thrown: Volume 142cafcf-7e8a-4ed4-bdee-48cdd89ee04d could not be found. Jul 19 11:10:34.826531 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-e721150d-8d0d-45e2-a575-e605897071b5 tempest-VolumesNegativeTest-790337672 tempest-VolumesNegativeTest-790337672-project] Request method failure captured: Jul 19 11:10:34.826531 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: request: GET /compute/v2.1/os-volumes/142cafcf-7e8a-4ed4-bdee-48cdd89ee04d HTTP/1.1 Jul 19 11:10:34.826531 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept: application/json Jul 19 11:10:34.826531 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept-Encoding: identity Jul 19 11:10:34.826531 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Connection: close Jul 19 11:10:34.826531 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Length: 0 Jul 19 11:10:34.826531 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Type: application/json Jul 19 11:10:34.826531 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Host: 10.222.0.33 Jul 19 11:10:34.826531 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Openstack-System-Scope: None Jul 19 11:10:34.826531 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: User-Agent: python-urllib3/1.26.9 Jul 19 11:10:34.826531 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Auth-Token: gAAAAABi1pEqJQH-2K-crnGbryAI8N_9iTtzsh_CJ_MXIFbrtpdGD8DfmTWxfnwy-deR4MrZo4Cm_RTgIQURemxGkOMXk9BPJ5kl2lgxPBRvldajb5NgAs-bA-3t_g2HJboxY0mOF2NX6cX0YWZl923qjv1RKXiOQuhlUE_KIWxDtPSJCaImagg Jul 19 11:10:34.826531 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Id: None Jul 19 11:10:34.826531 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Name: None Jul 19 11:10:34.826531 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Identity-Status: Confirmed Jul 19 11:10:34.826531 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Is-Admin-Project: True Jul 19 11:10:34.826531 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Id: default Jul 19 11:10:34.826531 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Name: Default Jul 19 11:10:34.826531 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Id: d916fb4d05694bd5ba3f9d61678fba03 Jul 19 11:10:34.826531 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Name: tempest-VolumesNegativeTest-790337672 Jul 19 11:10:34.826531 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Role: member,reader Jul 19 11:10:34.826531 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Roles: member,reader Jul 19 11:10:34.828000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/d916fb4d05694bd5ba3f9d61678fba03"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/d916fb4d05694bd5ba3f9d61678fba03"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/d916fb4d05694bd5ba3f9d61678fba03"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_d916fb4d05694bd5ba3f9d61678fba03"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:10:34.828000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant: tempest-VolumesNegativeTest-790337672 Jul 19 11:10:34.828000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Id: d916fb4d05694bd5ba3f9d61678fba03 Jul 19 11:10:34.828000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Name: tempest-VolumesNegativeTest-790337672 Jul 19 11:10:34.828000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User: tempest-VolumesNegativeTest-790337672-project Jul 19 11:10:34.828000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Id: default Jul 19 11:10:34.828000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Name: Default Jul 19 11:10:34.828000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Id: 5f95631bc447497ba062a57c877a133b Jul 19 11:10:34.828000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Name: tempest-VolumesNegativeTest-790337672-project Jul 19 11:10:34.828000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: method: Jul 19 11:10:34.828000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: exception: Volume 142cafcf-7e8a-4ed4-bdee-48cdd89ee04d could not be found. Jul 19 11:10:34.828000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: action_args: {'id': '142cafcf-7e8a-4ed4-bdee-48cdd89ee04d'} Jul 19 11:10:34.828000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:10:34.828953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:34.828953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 19 11:10:34.828953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 19 11:10:34.828953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 19 11:10:34.828953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 19 11:10:34.828953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 19 11:10:34.828953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 19 11:10:34.828953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 19 11:10:34.828953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 19 11:10:34.828953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 19 11:10:34.828953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 19 11:10:34.828953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 19 11:10:34.828953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 19 11:10:34.828953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 19 11:10:34.828953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 19 11:10:34.828953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 142cafcf-7e8a-4ed4-bdee-48cdd89ee04d could not be found. (HTTP 404) (Request-ID: req-054302f6-4250-4bc7-ae39-795980d8dc62) Jul 19 11:10:34.828953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.828953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:10:34.828953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.830286 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:34.830286 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Jul 19 11:10:34.830286 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Jul 19 11:10:34.830286 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 19 11:10:34.830286 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 19 11:10:34.830286 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 19 11:10:34.830286 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 19 11:10:34.830286 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 19 11:10:34.830286 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 19 11:10:34.830286 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 19 11:10:34.830286 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 19 11:10:34.830286 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 19 11:10:34.830286 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 19 11:10:34.830286 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 19 11:10:34.830286 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 19 11:10:34.830286 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 19 11:10:34.830286 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 19 11:10:34.830286 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 19 11:10:34.830286 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 19 11:10:34.831683 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 19 11:10:34.831683 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 19 11:10:34.831683 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 19 11:10:34.831683 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 19 11:10:34.831683 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 142cafcf-7e8a-4ed4-bdee-48cdd89ee04d could not be found. Jul 19 11:10:34.831683 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.831683 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:10:34.831683 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.831683 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:34.831683 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:10:34.831683 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:10:34.831683 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:10:34.831683 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:10:34.831683 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 19 11:10:34.831683 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 19 11:10:34.831683 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:10:34.831683 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:10:34.831683 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Jul 19 11:10:34.831683 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:10:34.833038 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 142cafcf-7e8a-4ed4-bdee-48cdd89ee04d could not be found. Jul 19 11:10:34.833038 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.833038 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:10:34.833038 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.833038 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:34.833038 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:10:34.833038 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:10:34.833038 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:10:34.833038 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:10:34.833038 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 142cafcf-7e8a-4ed4-bdee-48cdd89ee04d could not be found. Jul 19 11:10:34.833038 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:34.833038 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-e721150d-8d0d-45e2-a575-e605897071b5 tempest-VolumesNegativeTest-790337672 tempest-VolumesNegativeTest-790337672-project] Returning 404 to user: Volume 142cafcf-7e8a-4ed4-bdee-48cdd89ee04d could not be found. {{(pid=117848) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:10:34.833038 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-e721150d-8d0d-45e2-a575-e605897071b5 tempest-VolumesNegativeTest-790337672 tempest-VolumesNegativeTest-790337672-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/142cafcf-7e8a-4ed4-bdee-48cdd89ee04d" status: 404 len: 109 microversion: 2.1 time: 0.034847 Jul 19 11:10:34.833038 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 470/917] 10.222.0.33 () {58 vars in 1244 bytes} [Tue Jul 19 11:10:34 2022] GET /compute/v2.1/os-volumes/142cafcf-7e8a-4ed4-bdee-48cdd89ee04d => generated 109 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:10:35.061476 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-5c96d59e-da46-4e8e-b97b-892938f44047 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:10:35.062640 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-5c96d59e-da46-4e8e-b97b-892938f44047 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:35.068618 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5c96d59e-da46-4e8e-b97b-892938f44047 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:35.069292 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5c96d59e-da46-4e8e-b97b-892938f44047 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:35.070052 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5c96d59e-da46-4e8e-b97b-892938f44047 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:35.070485 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5c96d59e-da46-4e8e-b97b-892938f44047 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:35.094838 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.objects.instance [None req-5c96d59e-da46-4e8e-b97b-892938f44047 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lazy-loading 'flavor' on Instance uuid 36abf2ae-3b4c-4095-b9ba-2bd738e266d9 {{(pid=117849) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 19 11:10:35.257986 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-c4284042-2820-43ee-9c58-3a3473bc078f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-1499670947"}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:10:35.259775 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-c4284042-2820-43ee-9c58-3a3473bc078f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:35.261979 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_api:SELECT=67,nova_api:UPDATE=1 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:10:35.269747 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c4284042-2820-43ee-9c58-3a3473bc078f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:35.270451 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c4284042-2820-43ee-9c58-3a3473bc078f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:35.271232 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c4284042-2820-43ee-9c58-3a3473bc078f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:35.271747 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c4284042-2820-43ee-9c58-3a3473bc078f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:35.382926 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-5c96d59e-da46-4e8e-b97b-892938f44047 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "POST /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9/action" status: 202 len: 0 microversion: 2.20 time: 0.325328 Jul 19 11:10:35.383544 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 448/918] 10.222.0.33 () {62 vars in 1318 bytes} [Tue Jul 19 11:10:35 2022] POST /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9/action => generated 0 bytes in 327 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 19 11:10:35.392396 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-94c36ed3-9dea-4dfe-b382-739436961eb1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:35.393385 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-94c36ed3-9dea-4dfe-b382-739436961eb1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:35.402890 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-94c36ed3-9dea-4dfe-b382-739436961eb1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:35.405745 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-94c36ed3-9dea-4dfe-b382-739436961eb1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:35.406517 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-94c36ed3-9dea-4dfe-b382-739436961eb1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:35.406960 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-94c36ed3-9dea-4dfe-b382-739436961eb1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:35.438578 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.network.security_group_api [None req-c4284042-2820-43ee-9c58-3a3473bc078f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Removing security group 0c90f2f3-5e80-4c64-b49b-9025a5d2bb48 from port 6b6d7587-f2e5-4c60-8cf8-d1103dc5c0b6 Jul 19 11:10:35.569936 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-94c36ed3-9dea-4dfe-b382-739436961eb1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:35.573665 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=93,nova_cell1:UPDATE=4,nova_cell1:INSERT=3 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:10:35.579753 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-94c36ed3-9dea-4dfe-b382-739436961eb1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:35.581527 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-94c36ed3-9dea-4dfe-b382-739436961eb1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:35.582667 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-94c36ed3-9dea-4dfe-b382-739436961eb1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1723 microversion: 2.20 time: 0.194923 Jul 19 11:10:35.583374 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 449/919] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:10:35 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1723 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:10:36.071129 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-c4284042-2820-43ee-9c58-3a3473bc078f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "POST /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a/action" status: 202 len: 0 microversion: 2.1 time: 0.817569 Jul 19 11:10:36.077995 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 471/920] 10.222.0.33 () {60 vars in 1277 bytes} [Tue Jul 19 11:10:35 2022] POST /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a/action => generated 0 bytes in 822 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 19 11:10:36.419344 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-93daa9b3-7492-463f-9296-d93bdaf7c324 req-8eefc745-7bdc-447c-bb80-bf091372a7e9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "03624642-ce02-450f-821d-1c135e5d7a4a", "tag": "6b6d7587-f2e5-4c60-8cf8-d1103dc5c0b6"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:10:36.431502 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-93daa9b3-7492-463f-9296-d93bdaf7c324 req-8eefc745-7bdc-447c-bb80-bf091372a7e9 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:36.431952 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-93daa9b3-7492-463f-9296-d93bdaf7c324 req-8eefc745-7bdc-447c-bb80-bf091372a7e9 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:36.461927 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-93daa9b3-7492-463f-9296-d93bdaf7c324 req-8eefc745-7bdc-447c-bb80-bf091372a7e9 service nova] Creating event network-changed:6b6d7587-f2e5-4c60-8cf8-d1103dc5c0b6 for instance 03624642-ce02-450f-821d-1c135e5d7a4a on hv-cinder-maste Jul 19 11:10:36.468731 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-93daa9b3-7492-463f-9296-d93bdaf7c324 req-8eefc745-7bdc-447c-bb80-bf091372a7e9 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.393431 Jul 19 11:10:36.469928 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 450/921] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:10:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:10:36.605069 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-ae6172f1-d6ea-445b-8f2f-5957d31eb327 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:36.605757 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-ae6172f1-d6ea-445b-8f2f-5957d31eb327 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:36.619756 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ae6172f1-d6ea-445b-8f2f-5957d31eb327 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:36.620285 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ae6172f1-d6ea-445b-8f2f-5957d31eb327 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:36.621125 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ae6172f1-d6ea-445b-8f2f-5957d31eb327 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:36.621921 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ae6172f1-d6ea-445b-8f2f-5957d31eb327 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:36.696884 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-fc70225d-be85-415d-968c-8665845aa6bf tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:36.697893 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-fc70225d-be85-415d-968c-8665845aa6bf tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:36.705416 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fc70225d-be85-415d-968c-8665845aa6bf tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:36.706307 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fc70225d-be85-415d-968c-8665845aa6bf tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:36.707513 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fc70225d-be85-415d-968c-8665845aa6bf tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:36.708111 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fc70225d-be85-415d-968c-8665845aa6bf tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:36.739858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-fc70225d-be85-415d-968c-8665845aa6bf tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.047768 Jul 19 11:10:36.745437 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 451/922] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:10:36 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487/os-volume_attachments => generated 25 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Jul 19 11:10:36.819469 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-110fffea-d2e0-4e68-a361-66fb16015a64 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:36.820369 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-110fffea-d2e0-4e68-a361-66fb16015a64 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:36.831263 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-110fffea-d2e0-4e68-a361-66fb16015a64 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:36.831925 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-110fffea-d2e0-4e68-a361-66fb16015a64 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:36.833532 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-110fffea-d2e0-4e68-a361-66fb16015a64 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:36.834155 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-110fffea-d2e0-4e68-a361-66fb16015a64 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:36.863624 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-110fffea-d2e0-4e68-a361-66fb16015a64 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Going to try to terminate instance {{(pid=117849) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 19 11:10:36.892671 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-ae6172f1-d6ea-445b-8f2f-5957d31eb327 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:36.902217 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-ae6172f1-d6ea-445b-8f2f-5957d31eb327 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:36.903741 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-ae6172f1-d6ea-445b-8f2f-5957d31eb327 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:36.905538 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-ae6172f1-d6ea-445b-8f2f-5957d31eb327 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1723 microversion: 2.20 time: 0.307158 Jul 19 11:10:36.906531 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 472/923] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:10:36 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1723 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:10:36.971616 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-110fffea-d2e0-4e68-a361-66fb16015a64 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "DELETE /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 204 len: 0 microversion: 2.1 time: 0.156236 Jul 19 11:10:36.972499 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 452/924] 10.222.0.33 () {58 vars in 1238 bytes} [Tue Jul 19 11:10:36 2022] DELETE /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 19 11:10:36.981102 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-60ea5cd1-da49-45da-bba7-6f881e173c3d tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:36.981964 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-60ea5cd1-da49-45da-bba7-6f881e173c3d tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:36.988671 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-60ea5cd1-da49-45da-bba7-6f881e173c3d tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:36.989478 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-60ea5cd1-da49-45da-bba7-6f881e173c3d tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:36.990575 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-60ea5cd1-da49-45da-bba7-6f881e173c3d tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:36.991171 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-60ea5cd1-da49-45da-bba7-6f881e173c3d tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:37.260466 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-60ea5cd1-da49-45da-bba7-6f881e173c3d tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:37.270365 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-60ea5cd1-da49-45da-bba7-6f881e173c3d tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1673 microversion: 2.1 time: 0.293473 Jul 19 11:10:37.271317 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 473/925] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:36 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1673 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:10:37.541473 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-6a600ad1-6a23-433c-9333-e83980f4ff5c tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:37.542045 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-6a600ad1-6a23-433c-9333-e83980f4ff5c tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:37.548931 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6a600ad1-6a23-433c-9333-e83980f4ff5c tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:37.549374 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6a600ad1-6a23-433c-9333-e83980f4ff5c tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:37.550262 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6a600ad1-6a23-433c-9333-e83980f4ff5c tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:37.550719 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6a600ad1-6a23-433c-9333-e83980f4ff5c tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:37.634542 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.objects.instance [None req-6a600ad1-6a23-433c-9333-e83980f4ff5c tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lazy-loading 'services' on Instance uuid 03624642-ce02-450f-821d-1c135e5d7a4a {{(pid=117849) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 19 11:10:37.720042 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-6a600ad1-6a23-433c-9333-e83980f4ff5c tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "DELETE /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a/os-volume_attachments/4be186cd-8f4e-4623-ba8c-2bc3eb5565b6" status: 202 len: 0 microversion: 2.1 time: 0.183136 Jul 19 11:10:37.724269 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 453/926] 10.222.0.33 () {58 vars in 1415 bytes} [Tue Jul 19 11:10:37 2022] DELETE /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a/os-volume_attachments/4be186cd-8f4e-4623-ba8c-2bc3eb5565b6 => generated 0 bytes in 188 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 19 11:10:37.729301 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-caaf8fb1-4c4c-4924-8691-046199d7d7fe tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:37.730559 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-caaf8fb1-4c4c-4924-8691-046199d7d7fe tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:37.737606 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-caaf8fb1-4c4c-4924-8691-046199d7d7fe tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:37.738063 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-caaf8fb1-4c4c-4924-8691-046199d7d7fe tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:37.738895 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-caaf8fb1-4c4c-4924-8691-046199d7d7fe tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:37.739337 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-caaf8fb1-4c4c-4924-8691-046199d7d7fe tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:37.773425 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-caaf8fb1-4c4c-4924-8691-046199d7d7fe tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047733 Jul 19 11:10:37.777388 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 474/927] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:10:37 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:10:37.927536 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-53712a58-e2a7-4034-ba66-4679fc4e52ae tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:37.928231 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-53712a58-e2a7-4034-ba66-4679fc4e52ae tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:37.937974 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-53712a58-e2a7-4034-ba66-4679fc4e52ae tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:37.938473 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-53712a58-e2a7-4034-ba66-4679fc4e52ae tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:37.939305 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-53712a58-e2a7-4034-ba66-4679fc4e52ae tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:37.939735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-53712a58-e2a7-4034-ba66-4679fc4e52ae tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:38.087606 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-53712a58-e2a7-4034-ba66-4679fc4e52ae tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:38.096324 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-53712a58-e2a7-4034-ba66-4679fc4e52ae tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:38.098157 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-53712a58-e2a7-4034-ba66-4679fc4e52ae tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:38.099378 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-53712a58-e2a7-4034-ba66-4679fc4e52ae tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1723 microversion: 2.20 time: 0.176794 Jul 19 11:10:38.099939 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 454/928] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:10:37 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1723 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:10:38.288957 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-25b366f2-429f-49fb-9913-3e010e4b6a88 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:38.289522 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-25b366f2-429f-49fb-9913-3e010e4b6a88 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:38.296138 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-25b366f2-429f-49fb-9913-3e010e4b6a88 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:38.296887 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-25b366f2-429f-49fb-9913-3e010e4b6a88 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:38.297655 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-25b366f2-429f-49fb-9913-3e010e4b6a88 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:38.298280 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-25b366f2-429f-49fb-9913-3e010e4b6a88 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:38.450410 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-25b366f2-429f-49fb-9913-3e010e4b6a88 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:38.458669 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-25b366f2-429f-49fb-9913-3e010e4b6a88 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1673 microversion: 2.1 time: 0.174400 Jul 19 11:10:38.459235 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 475/929] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:38 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1673 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:38.485970 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-7194485c-775a-4e1b-8307-feb40170590e req-5025aba3-4332-487a-9379-a4b73e3fde95 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "03624642-ce02-450f-821d-1c135e5d7a4a", "tag": "6b6d7587-f2e5-4c60-8cf8-d1103dc5c0b6"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:10:38.495375 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-7194485c-775a-4e1b-8307-feb40170590e req-5025aba3-4332-487a-9379-a4b73e3fde95 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:38.495916 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-7194485c-775a-4e1b-8307-feb40170590e req-5025aba3-4332-487a-9379-a4b73e3fde95 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:38.510110 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-7194485c-775a-4e1b-8307-feb40170590e req-5025aba3-4332-487a-9379-a4b73e3fde95 service nova] Creating event network-changed:6b6d7587-f2e5-4c60-8cf8-d1103dc5c0b6 for instance 03624642-ce02-450f-821d-1c135e5d7a4a on hv-cinder-maste Jul 19 11:10:38.517286 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-7194485c-775a-4e1b-8307-feb40170590e req-5025aba3-4332-487a-9379-a4b73e3fde95 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033639 Jul 19 11:10:38.520179 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 455/930] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:10:38 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:10:38.784266 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-09d275c8-92ce-4709-8674-deacc5b95b9d tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:38.785335 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-09d275c8-92ce-4709-8674-deacc5b95b9d tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:38.791661 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-09d275c8-92ce-4709-8674-deacc5b95b9d tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:38.792107 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-09d275c8-92ce-4709-8674-deacc5b95b9d tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:38.792912 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-09d275c8-92ce-4709-8674-deacc5b95b9d tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:38.793344 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-09d275c8-92ce-4709-8674-deacc5b95b9d tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:38.822860 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-09d275c8-92ce-4709-8674-deacc5b95b9d tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042503 Jul 19 11:10:38.823608 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 476/931] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:10:38 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:10:39.125291 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-db71eb00-6582-4922-97c4-6198ab8813cf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:39.126075 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-db71eb00-6582-4922-97c4-6198ab8813cf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:39.132728 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-db71eb00-6582-4922-97c4-6198ab8813cf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:39.133265 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-db71eb00-6582-4922-97c4-6198ab8813cf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:39.134143 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-db71eb00-6582-4922-97c4-6198ab8813cf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:39.134662 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-db71eb00-6582-4922-97c4-6198ab8813cf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:39.273800 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-db71eb00-6582-4922-97c4-6198ab8813cf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:39.282065 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-db71eb00-6582-4922-97c4-6198ab8813cf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:39.283555 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-db71eb00-6582-4922-97c4-6198ab8813cf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:39.284702 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-db71eb00-6582-4922-97c4-6198ab8813cf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1723 microversion: 2.20 time: 0.170928 Jul 19 11:10:39.285261 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 456/932] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:10:39 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1723 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:10:39.478500 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-03b1df7b-6389-4263-8606-8c6a78b67a1a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:39.479171 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-03b1df7b-6389-4263-8606-8c6a78b67a1a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:39.487934 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-03b1df7b-6389-4263-8606-8c6a78b67a1a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:39.488828 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-03b1df7b-6389-4263-8606-8c6a78b67a1a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:39.489915 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-03b1df7b-6389-4263-8606-8c6a78b67a1a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:39.490508 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-03b1df7b-6389-4263-8606-8c6a78b67a1a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:39.629280 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-03b1df7b-6389-4263-8606-8c6a78b67a1a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:39.637149 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-03b1df7b-6389-4263-8606-8c6a78b67a1a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1673 microversion: 2.1 time: 0.164107 Jul 19 11:10:39.637719 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 477/933] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:39 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1673 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:39.834945 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-dabecc10-8e5c-473b-9eec-95c5bd68da13 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:39.835885 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-dabecc10-8e5c-473b-9eec-95c5bd68da13 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:39.841948 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dabecc10-8e5c-473b-9eec-95c5bd68da13 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:39.842893 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dabecc10-8e5c-473b-9eec-95c5bd68da13 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:39.843655 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dabecc10-8e5c-473b-9eec-95c5bd68da13 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:39.844097 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dabecc10-8e5c-473b-9eec-95c5bd68da13 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:39.872877 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-dabecc10-8e5c-473b-9eec-95c5bd68da13 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041798 Jul 19 11:10:39.873480 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 457/934] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:10:39 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:10:40.303027 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-9586b508-8185-4bf7-8fe6-3f8e3ad53608 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:40.303635 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-9586b508-8185-4bf7-8fe6-3f8e3ad53608 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:40.317922 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9586b508-8185-4bf7-8fe6-3f8e3ad53608 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:40.318593 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9586b508-8185-4bf7-8fe6-3f8e3ad53608 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:40.319534 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9586b508-8185-4bf7-8fe6-3f8e3ad53608 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:40.321258 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9586b508-8185-4bf7-8fe6-3f8e3ad53608 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:40.331624 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=100 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:10:40.578514 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-9586b508-8185-4bf7-8fe6-3f8e3ad53608 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:40.587469 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-9586b508-8185-4bf7-8fe6-3f8e3ad53608 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:40.589244 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-9586b508-8185-4bf7-8fe6-3f8e3ad53608 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:40.590400 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-9586b508-8185-4bf7-8fe6-3f8e3ad53608 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1723 microversion: 2.20 time: 0.291299 Jul 19 11:10:40.590955 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 478/935] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:10:40 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1723 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:10:40.654407 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-db6027a0-7e62-4980-be6e-ec3b106d4fa0 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:40.655025 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-db6027a0-7e62-4980-be6e-ec3b106d4fa0 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:40.661569 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-db6027a0-7e62-4980-be6e-ec3b106d4fa0 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:40.662017 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-db6027a0-7e62-4980-be6e-ec3b106d4fa0 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:40.662806 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-db6027a0-7e62-4980-be6e-ec3b106d4fa0 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:40.663237 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-db6027a0-7e62-4980-be6e-ec3b106d4fa0 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:40.728175 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-db6027a0-7e62-4980-be6e-ec3b106d4fa0 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:40.737392 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-db6027a0-7e62-4980-be6e-ec3b106d4fa0 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 200 len: 1605 microversion: 2.1 time: 0.086821 Jul 19 11:10:40.748638 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 458/936] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:40 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 1605 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:40.888894 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-bde30e3a-29ca-4d91-986f-c6aaf3b3afaa tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:40.890411 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-2996072a-5440-42b4-a22e-8e9c0db8a62b req-6e29f7ef-4e7d-4f40-8aab-feabaaaae160 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "940e8a5c-b365-40c2-8361-fbb677969487", "name": "network-vif-deleted", "tag": "d928944f-b596-4faa-b70f-efdde2d1f07e"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:10:40.894072 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-bde30e3a-29ca-4d91-986f-c6aaf3b3afaa tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:40.908072 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-2996072a-5440-42b4-a22e-8e9c0db8a62b req-6e29f7ef-4e7d-4f40-8aab-feabaaaae160 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:40.908520 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-2996072a-5440-42b4-a22e-8e9c0db8a62b req-6e29f7ef-4e7d-4f40-8aab-feabaaaae160 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:40.909082 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bde30e3a-29ca-4d91-986f-c6aaf3b3afaa tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:40.909753 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bde30e3a-29ca-4d91-986f-c6aaf3b3afaa tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:40.910318 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bde30e3a-29ca-4d91-986f-c6aaf3b3afaa tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:40.910756 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bde30e3a-29ca-4d91-986f-c6aaf3b3afaa tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:40.939567 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-2996072a-5440-42b4-a22e-8e9c0db8a62b req-6e29f7ef-4e7d-4f40-8aab-feabaaaae160 service nova] Creating event network-vif-deleted:d928944f-b596-4faa-b70f-efdde2d1f07e for instance 940e8a5c-b365-40c2-8361-fbb677969487 on hv-cinder-maste Jul 19 11:10:40.948374 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-2996072a-5440-42b4-a22e-8e9c0db8a62b req-6e29f7ef-4e7d-4f40-8aab-feabaaaae160 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.070177 Jul 19 11:10:40.952572 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 479/937] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:10:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:10:40.967605 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-bde30e3a-29ca-4d91-986f-c6aaf3b3afaa tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.085824 Jul 19 11:10:40.968781 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 459/938] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:10:40 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a/os-volume_attachments => generated 25 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 19 11:10:40.979412 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-3f319e36-4061-46a5-a010-412261ce0fb4 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:40.980369 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-3f319e36-4061-46a5-a010-412261ce0fb4 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:40.988606 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3f319e36-4061-46a5-a010-412261ce0fb4 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:40.989370 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3f319e36-4061-46a5-a010-412261ce0fb4 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:40.990364 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3f319e36-4061-46a5-a010-412261ce0fb4 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:40.990884 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3f319e36-4061-46a5-a010-412261ce0fb4 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:41.091007 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.wsgi [None req-3f319e36-4061-46a5-a010-412261ce0fb4 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] HTTP exception thrown: Instance 03624642-ce02-450f-821d-1c135e5d7a4a is not attached to volume 4be186cd-8f4e-4623-ba8c-2bc3eb5565b6 Jul 19 11:10:41.093685 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-3f319e36-4061-46a5-a010-412261ce0fb4 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Request method failure captured: Jul 19 11:10:41.093685 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: request: DELETE /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a/os-volume_attachments/4be186cd-8f4e-4623-ba8c-2bc3eb5565b6 HTTP/1.1 Jul 19 11:10:41.093685 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept: application/json Jul 19 11:10:41.093685 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept-Encoding: identity Jul 19 11:10:41.093685 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Connection: close Jul 19 11:10:41.093685 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Length: 0 Jul 19 11:10:41.093685 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Type: application/json Jul 19 11:10:41.093685 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Host: 10.222.0.33 Jul 19 11:10:41.093685 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Openstack-System-Scope: None Jul 19 11:10:41.093685 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: User-Agent: python-urllib3/1.26.9 Jul 19 11:10:41.093685 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Auth-Token: gAAAAABi1pBqAQgKP5JyqcSTFiCeDPefAr26MUDPwxC3QXjPMYdp-MQklgz7HUR3fQSDcG9Si_QsMsJMAETbfQY7V9tN_aGZopZGDrOM9TTeR0tqssOhwFOYdDz6Te6fP1Zx6wlE_NJYQME2cqzGdLjiJX-9MS88De0sKIoUnxnUGcAJkGupjC4 Jul 19 11:10:41.093685 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Id: None Jul 19 11:10:41.093685 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Name: None Jul 19 11:10:41.093685 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Identity-Status: Confirmed Jul 19 11:10:41.093685 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Is-Admin-Project: True Jul 19 11:10:41.093685 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Id: default Jul 19 11:10:41.093685 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Name: Default Jul 19 11:10:41.093685 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Id: 05630eeaf84a4d1bafd213d419f7a97a Jul 19 11:10:41.093685 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Name: tempest-TestMinimumBasicScenario-1471164354 Jul 19 11:10:41.093685 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Role: member,reader Jul 19 11:10:41.093685 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Roles: member,reader Jul 19 11:10:41.095050 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/05630eeaf84a4d1bafd213d419f7a97a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/05630eeaf84a4d1bafd213d419f7a97a"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/05630eeaf84a4d1bafd213d419f7a97a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_05630eeaf84a4d1bafd213d419f7a97a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:10:41.095050 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant: tempest-TestMinimumBasicScenario-1471164354 Jul 19 11:10:41.095050 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Id: 05630eeaf84a4d1bafd213d419f7a97a Jul 19 11:10:41.095050 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1471164354 Jul 19 11:10:41.095050 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User: tempest-TestMinimumBasicScenario-1471164354-project Jul 19 11:10:41.095050 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Id: default Jul 19 11:10:41.095050 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Name: Default Jul 19 11:10:41.095050 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Id: 9e853b49c2c744e89243ec51e7a1a379 Jul 19 11:10:41.095050 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Name: tempest-TestMinimumBasicScenario-1471164354-project Jul 19 11:10:41.095050 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: method: > Jul 19 11:10:41.095050 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: exception: Instance 03624642-ce02-450f-821d-1c135e5d7a4a is not attached to volume 4be186cd-8f4e-4623-ba8c-2bc3eb5565b6 Jul 19 11:10:41.095734 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: action_args: {'server_id': '03624642-ce02-450f-821d-1c135e5d7a4a', 'id': '4be186cd-8f4e-4623-ba8c-2bc3eb5565b6'} Jul 19 11:10:41.095734 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:10:41.095734 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:41.095734 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Jul 19 11:10:41.095734 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Jul 19 11:10:41.095734 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 19 11:10:41.095734 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 19 11:10:41.095734 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 281, in get_by_volume_and_instance Jul 19 11:10:41.095734 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Jul 19 11:10:41.095734 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 4be186cd-8f4e-4623-ba8c-2bc3eb5565b6. Jul 19 11:10:41.095734 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:41.095734 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:10:41.095734 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:41.095734 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:41.095734 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:10:41.095734 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:10:41.095734 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:10:41.095734 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:10:41.095734 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:10:41.097009 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:10:41.097009 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Jul 19 11:10:41.097009 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Jul 19 11:10:41.097009 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 03624642-ce02-450f-821d-1c135e5d7a4a is not attached to volume 4be186cd-8f4e-4623-ba8c-2bc3eb5565b6 Jul 19 11:10:41.097009 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:41.097009 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:10:41.097009 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:41.097009 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:41.097009 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:10:41.097009 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:10:41.097009 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:10:41.097009 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:10:41.097009 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 03624642-ce02-450f-821d-1c135e5d7a4a is not attached to volume 4be186cd-8f4e-4623-ba8c-2bc3eb5565b6 Jul 19 11:10:41.097009 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:41.097009 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-3f319e36-4061-46a5-a010-412261ce0fb4 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Returning 404 to user: Instance 03624642-ce02-450f-821d-1c135e5d7a4a is not attached to volume 4be186cd-8f4e-4623-ba8c-2bc3eb5565b6 {{(pid=117848) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:10:41.098775 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-3f319e36-4061-46a5-a010-412261ce0fb4 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "DELETE /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a/os-volume_attachments/4be186cd-8f4e-4623-ba8c-2bc3eb5565b6" status: 404 len: 154 microversion: 2.1 time: 0.120651 Jul 19 11:10:41.111996 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 480/939] 10.222.0.33 () {58 vars in 1415 bytes} [Tue Jul 19 11:10:40 2022] DELETE /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a/os-volume_attachments/4be186cd-8f4e-4623-ba8c-2bc3eb5565b6 => generated 154 bytes in 137 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:10:41.610880 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-47c49da2-9d70-49d2-b6b2-f2ffa28a9bea tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:41.611604 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-47c49da2-9d70-49d2-b6b2-f2ffa28a9bea tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:41.617685 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-47c49da2-9d70-49d2-b6b2-f2ffa28a9bea tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:41.618213 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-47c49da2-9d70-49d2-b6b2-f2ffa28a9bea tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:41.619386 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-47c49da2-9d70-49d2-b6b2-f2ffa28a9bea tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:41.619891 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-47c49da2-9d70-49d2-b6b2-f2ffa28a9bea tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:41.756139 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-7a42b6b0-f13d-4852-ad13-518570795385 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:41.756772 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-7a42b6b0-f13d-4852-ad13-518570795385 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:41.761856 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-47c49da2-9d70-49d2-b6b2-f2ffa28a9bea tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:41.769705 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-47c49da2-9d70-49d2-b6b2-f2ffa28a9bea tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:41.770052 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7a42b6b0-f13d-4852-ad13-518570795385 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:41.771033 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7a42b6b0-f13d-4852-ad13-518570795385 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:41.772048 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-47c49da2-9d70-49d2-b6b2-f2ffa28a9bea tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:41.773250 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7a42b6b0-f13d-4852-ad13-518570795385 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:41.773705 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7a42b6b0-f13d-4852-ad13-518570795385 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:41.773960 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-47c49da2-9d70-49d2-b6b2-f2ffa28a9bea tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1723 microversion: 2.20 time: 0.168703 Jul 19 11:10:41.775143 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 460/940] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:10:41 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1723 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:10:41.791594 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.wsgi [None req-7a42b6b0-f13d-4852-ad13-518570795385 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] HTTP exception thrown: Instance 940e8a5c-b365-40c2-8361-fbb677969487 could not be found. Jul 19 11:10:41.793189 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-7a42b6b0-f13d-4852-ad13-518570795385 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Request method failure captured: Jul 19 11:10:41.793189 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: request: GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 HTTP/1.1 Jul 19 11:10:41.793189 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept: application/json Jul 19 11:10:41.793189 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept-Encoding: identity Jul 19 11:10:41.793189 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Connection: close Jul 19 11:10:41.793189 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Length: 0 Jul 19 11:10:41.793189 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Type: application/json Jul 19 11:10:41.793189 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Host: 10.222.0.33 Jul 19 11:10:41.793189 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Openstack-System-Scope: None Jul 19 11:10:41.793189 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: User-Agent: python-urllib3/1.26.9 Jul 19 11:10:41.793189 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Auth-Token: gAAAAABi1pAu99_KxkXaZby1psxJ1ftONI49OqTcD4XzMi6hyP2ZiznBv9HGO6U0N-i4E5yVLAsyrvg4F0nQOhLM8vQUGcVey3N6L-tk-5qYOymaTM2Q0YoDwJ2T5cyt9gxJB_YaQKNDCdDnlwJK6XuChZuA6B6r4ILZAuFVdn5hixAgJPbdxDY Jul 19 11:10:41.793189 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Id: None Jul 19 11:10:41.793189 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Name: None Jul 19 11:10:41.793189 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Identity-Status: Confirmed Jul 19 11:10:41.793189 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Is-Admin-Project: True Jul 19 11:10:41.793189 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Id: default Jul 19 11:10:41.793189 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Name: Default Jul 19 11:10:41.793189 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Id: 54b5133f102c4606ac7981542949b178 Jul 19 11:10:41.793189 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Name: tempest-AttachVolumeTestJSON-815902756 Jul 19 11:10:41.793189 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Role: member,reader Jul 19 11:10:41.793189 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Roles: member,reader Jul 19 11:10:41.794665 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/54b5133f102c4606ac7981542949b178"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/54b5133f102c4606ac7981542949b178"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/54b5133f102c4606ac7981542949b178"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_54b5133f102c4606ac7981542949b178"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:10:41.794665 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant: tempest-AttachVolumeTestJSON-815902756 Jul 19 11:10:41.794665 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Id: 54b5133f102c4606ac7981542949b178 Jul 19 11:10:41.794665 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Name: tempest-AttachVolumeTestJSON-815902756 Jul 19 11:10:41.794665 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User: tempest-AttachVolumeTestJSON-815902756-project Jul 19 11:10:41.794665 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Id: default Jul 19 11:10:41.794665 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Name: Default Jul 19 11:10:41.794665 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Id: c0b6f4cfede44b2aa8f4a8dfc1e211d6 Jul 19 11:10:41.794665 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Name: tempest-AttachVolumeTestJSON-815902756-project Jul 19 11:10:41.794665 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: method: > Jul 19 11:10:41.794665 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: exception: Instance 940e8a5c-b365-40c2-8361-fbb677969487 could not be found. Jul 19 11:10:41.794665 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: action_args: {'id': '940e8a5c-b365-40c2-8361-fbb677969487'} Jul 19 11:10:41.796104 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:10:41.796104 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:41.796104 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:10:41.796104 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:10:41.796104 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:10:41.796104 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:10:41.796104 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:10:41.796104 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:10:41.796104 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:10:41.796104 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:10:41.796104 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 940e8a5c-b365-40c2-8361-fbb677969487 could not be found. Jul 19 11:10:41.796104 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:41.796104 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:10:41.796104 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:41.796104 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:41.796104 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:10:41.796104 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:10:41.796104 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:10:41.796104 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:10:41.796104 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:10:41.797885 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:10:41.797885 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 19 11:10:41.797885 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 19 11:10:41.797885 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:10:41.797885 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:10:41.797885 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:10:41.797885 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:10:41.797885 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 940e8a5c-b365-40c2-8361-fbb677969487 could not be found. Jul 19 11:10:41.797885 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:41.797885 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:10:41.797885 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:41.797885 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:41.797885 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:10:41.797885 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:10:41.797885 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:10:41.797885 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:10:41.797885 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 940e8a5c-b365-40c2-8361-fbb677969487 could not be found. Jul 19 11:10:41.797885 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:10:41.800107 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-7a42b6b0-f13d-4852-ad13-518570795385 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Returning 404 to user: Instance 940e8a5c-b365-40c2-8361-fbb677969487 could not be found. {{(pid=117848) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:10:41.801095 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-7a42b6b0-f13d-4852-ad13-518570795385 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 404 len: 111 microversion: 2.1 time: 0.049144 Jul 19 11:10:41.801569 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 481/941] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:41 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 111 bytes in 50 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:10:41.809913 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-92c2c9ac-1ae5-43d3-8617-f279b8bc6bd7 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '.version_select at 0x7f04aa1c7f70>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:41.817949 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_api:SELECT=78,nova_api:DELETE=1,nova_api:UPDATE=3 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:10:41.832632 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-92c2c9ac-1ae5-43d3-8617-f279b8bc6bd7 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1751795436" status: 202 len: 0 microversion: 2.1 time: 0.026275 Jul 19 11:10:41.837096 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 461/942] 10.222.0.33 () {58 vars in 1220 bytes} [Tue Jul 19 11:10:41 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1751795436 => generated 0 bytes in 32 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 19 11:10:42.257256 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-460c14c9-47a4-46e6-9683-2a4755408706 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:42.257836 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-460c14c9-47a4-46e6-9683-2a4755408706 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:42.264356 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-460c14c9-47a4-46e6-9683-2a4755408706 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:42.264842 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-460c14c9-47a4-46e6-9683-2a4755408706 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:42.265798 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-460c14c9-47a4-46e6-9683-2a4755408706 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:42.266454 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-460c14c9-47a4-46e6-9683-2a4755408706 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:42.298652 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-460c14c9-47a4-46e6-9683-2a4755408706 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Going to try to terminate instance {{(pid=117848) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 19 11:10:42.384442 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-460c14c9-47a4-46e6-9683-2a4755408706 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "DELETE /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 204 len: 0 microversion: 2.1 time: 0.130814 Jul 19 11:10:42.385054 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 482/943] 10.222.0.33 () {58 vars in 1238 bytes} [Tue Jul 19 11:10:42 2022] DELETE /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 0 bytes in 132 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 19 11:10:42.394909 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-2f06ac2a-332d-41d0-b7eb-d37fa1bd4e87 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:42.395497 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-2f06ac2a-332d-41d0-b7eb-d37fa1bd4e87 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:42.402230 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2f06ac2a-332d-41d0-b7eb-d37fa1bd4e87 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:42.402675 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2f06ac2a-332d-41d0-b7eb-d37fa1bd4e87 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:42.403430 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2f06ac2a-332d-41d0-b7eb-d37fa1bd4e87 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:42.404021 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2f06ac2a-332d-41d0-b7eb-d37fa1bd4e87 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:42.529941 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-3881bc96-6fac-4540-8370-433e99ec86fc tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Action: 'create', calling method: .version_select at 0x7f04aa495160>, body: {"keypair": {"name": "tempest-keypair-1910311538", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJFplU3R+Is8MJUOleMMAlYnozwqiqoGU6J3wEwceO5egdgci7w+3noowt/d58RLBJjFTM7PDPWjhilkYoAMAVxJALPI59TgoeTqjkVvXcI60P453RDB0VZlCWC6GJV/1w=="}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:10:42.543981 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.quota [None req-3881bc96-6fac-4540-8370-433e99ec86fc tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Getting quotas for project 54b5133f102c4606ac7981542949b178. Resources: {'key_pairs'} {{(pid=117848) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:10:42.547847 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.quota [None req-3881bc96-6fac-4540-8370-433e99ec86fc tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Getting quotas for user c0b6f4cfede44b2aa8f4a8dfc1e211d6 and project 54b5133f102c4606ac7981542949b178. Resources: {'key_pairs'} {{(pid=117848) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:10:42.548009 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-2f06ac2a-332d-41d0-b7eb-d37fa1bd4e87 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:42.555304 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-2f06ac2a-332d-41d0-b7eb-d37fa1bd4e87 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1554 microversion: 2.1 time: 0.165331 Jul 19 11:10:42.555869 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 462/944] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:42 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1554 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:42.570101 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-3881bc96-6fac-4540-8370-433e99ec86fc tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.044460 Jul 19 11:10:42.570769 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 483/945] 10.222.0.33 () {60 vars in 1158 bytes} [Tue Jul 19 11:10:42 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:10:42.793594 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-776cddb1-a9c6-4db3-a502-3ccf57aefd17 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:42.794215 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-776cddb1-a9c6-4db3-a502-3ccf57aefd17 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:42.801389 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-776cddb1-a9c6-4db3-a502-3ccf57aefd17 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:42.801833 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-776cddb1-a9c6-4db3-a502-3ccf57aefd17 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:42.802592 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-776cddb1-a9c6-4db3-a502-3ccf57aefd17 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:42.803024 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-776cddb1-a9c6-4db3-a502-3ccf57aefd17 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:42.952144 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-776cddb1-a9c6-4db3-a502-3ccf57aefd17 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:42.960407 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-776cddb1-a9c6-4db3-a502-3ccf57aefd17 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:42.962442 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-776cddb1-a9c6-4db3-a502-3ccf57aefd17 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:42.963785 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-776cddb1-a9c6-4db3-a502-3ccf57aefd17 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1723 microversion: 2.20 time: 0.174813 Jul 19 11:10:42.964140 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 463/946] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:10:42 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1723 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:10:43.574951 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-4e7fa174-4e50-4d7f-b51d-e4316fe534ab tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:43.576910 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-4e7fa174-4e50-4d7f-b51d-e4316fe534ab tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:43.584245 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4e7fa174-4e50-4d7f-b51d-e4316fe534ab tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:43.584907 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4e7fa174-4e50-4d7f-b51d-e4316fe534ab tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:43.585937 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4e7fa174-4e50-4d7f-b51d-e4316fe534ab tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:43.586600 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4e7fa174-4e50-4d7f-b51d-e4316fe534ab tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:43.775298 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-4e7fa174-4e50-4d7f-b51d-e4316fe534ab tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:43.784031 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-4e7fa174-4e50-4d7f-b51d-e4316fe534ab tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1554 microversion: 2.1 time: 0.214129 Jul 19 11:10:43.785358 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 484/947] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:43 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1554 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:43.901681 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-a067ac15-b0ab-4343-8586-00841475c88d tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1377354544", "imageRef": "3085883e-2289-4314-aa47-9c41d4368b02", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "f82d9bcd-5d24-448d-9b1c-e69a9c19f6e7"}], "security_groups": [{"name": "tempest-securitygroup--453241111"}], "key_name": "tempest-keypair-1910311538", "user_data": "}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:10:43.983045 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-6fdf0d4d-b2f8-44f9-9e50-3ca18527a8a9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:43.983977 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-6fdf0d4d-b2f8-44f9-9e50-3ca18527a8a9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:43.990215 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6fdf0d4d-b2f8-44f9-9e50-3ca18527a8a9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:43.990803 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6fdf0d4d-b2f8-44f9-9e50-3ca18527a8a9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:43.991687 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6fdf0d4d-b2f8-44f9-9e50-3ca18527a8a9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:43.992197 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6fdf0d4d-b2f8-44f9-9e50-3ca18527a8a9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:44.011425 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.network.neutron [None req-a067ac15-b0ab-4343-8586-00841475c88d tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] validate_networks() for [('f82d9bcd-5d24-448d-9b1c-e69a9c19f6e7', None, None, None, None, None)] {{(pid=117849) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 19 11:10:44.214097 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-6fdf0d4d-b2f8-44f9-9e50-3ca18527a8a9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:44.223637 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-6fdf0d4d-b2f8-44f9-9e50-3ca18527a8a9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:44.225923 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-6fdf0d4d-b2f8-44f9-9e50-3ca18527a8a9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:44.227598 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-6fdf0d4d-b2f8-44f9-9e50-3ca18527a8a9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1723 microversion: 2.20 time: 0.248711 Jul 19 11:10:44.228549 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 485/948] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:10:43 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1723 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:10:44.398952 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-a067ac15-b0ab-4343-8586-00841475c88d tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Flavor limits 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 19 11:10:44.399384 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-a067ac15-b0ab-4343-8586-00841475c88d tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Image limits 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 19 11:10:44.399859 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-a067ac15-b0ab-4343-8586-00841475c88d tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Flavor pref 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 19 11:10:44.400270 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-a067ac15-b0ab-4343-8586-00841475c88d tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Image pref 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 19 11:10:44.400681 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-a067ac15-b0ab-4343-8586-00841475c88d tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 19 11:10:44.410013 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a067ac15-b0ab-4343-8586-00841475c88d tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:44.410562 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a067ac15-b0ab-4343-8586-00841475c88d tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:44.411260 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a067ac15-b0ab-4343-8586-00841475c88d tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:44.411695 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a067ac15-b0ab-4343-8586-00841475c88d tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:44.432908 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-a067ac15-b0ab-4343-8586-00841475c88d tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Getting quotas for project 54b5133f102c4606ac7981542949b178. Resources: {'cores', 'instances', 'ram'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:10:44.440160 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-a067ac15-b0ab-4343-8586-00841475c88d tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Getting quotas for user c0b6f4cfede44b2aa8f4a8dfc1e211d6 and project 54b5133f102c4606ac7981542949b178. Resources: {'cores', 'instances', 'ram'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:10:44.454344 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-a067ac15-b0ab-4343-8586-00841475c88d tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Going to run 1 instances... {{(pid=117849) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 19 11:10:44.456644 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-a067ac15-b0ab-4343-8586-00841475c88d tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='3085883e-2289-4314-aa47-9c41d4368b02',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117849) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 19 11:10:44.508384 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-a067ac15-b0ab-4343-8586-00841475c88d tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.610512 Jul 19 11:10:44.509079 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 464/949] 10.222.0.33 () {60 vars in 1146 bytes} [Tue Jul 19 11:10:43 2022] POST /compute/v2.1/servers => generated 401 bytes in 612 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 19 11:10:44.519334 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-ccd52549-bda7-4c08-943f-a0d6a2e435ee tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:44.520029 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-ccd52549-bda7-4c08-943f-a0d6a2e435ee tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:44.582104 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-ccd52549-bda7-4c08-943f-a0d6a2e435ee tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:44.590035 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-ccd52549-bda7-4c08-943f-a0d6a2e435ee tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75" status: 200 len: 1238 microversion: 2.1 time: 0.074528 Jul 19 11:10:44.590916 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 486/950] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:44 2022] GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75 => generated 1238 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:44.803148 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-8e0e55da-f18b-43e3-b170-86791addf21f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:44.803910 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-8e0e55da-f18b-43e3-b170-86791addf21f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:44.812458 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8e0e55da-f18b-43e3-b170-86791addf21f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:44.813019 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8e0e55da-f18b-43e3-b170-86791addf21f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:44.813864 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8e0e55da-f18b-43e3-b170-86791addf21f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:44.814519 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8e0e55da-f18b-43e3-b170-86791addf21f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:45.027468 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-8e0e55da-f18b-43e3-b170-86791addf21f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:45.037370 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-8e0e55da-f18b-43e3-b170-86791addf21f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1554 microversion: 2.1 time: 0.238481 Jul 19 11:10:45.037933 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 465/951] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:44 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1554 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:45.168895 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-ca109543-3110-4ab5-b7ee-947a3bf3e561 req-cb4c1d9e-2154-46fc-b682-e864a65f81f4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "03624642-ce02-450f-821d-1c135e5d7a4a", "name": "network-vif-deleted", "tag": "6b6d7587-f2e5-4c60-8cf8-d1103dc5c0b6"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:10:45.178096 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-ca109543-3110-4ab5-b7ee-947a3bf3e561 req-cb4c1d9e-2154-46fc-b682-e864a65f81f4 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:45.178756 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-ca109543-3110-4ab5-b7ee-947a3bf3e561 req-cb4c1d9e-2154-46fc-b682-e864a65f81f4 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:45.189390 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-ca109543-3110-4ab5-b7ee-947a3bf3e561 req-cb4c1d9e-2154-46fc-b682-e864a65f81f4 service nova] Creating event network-vif-deleted:6b6d7587-f2e5-4c60-8cf8-d1103dc5c0b6 for instance 03624642-ce02-450f-821d-1c135e5d7a4a on hv-cinder-maste Jul 19 11:10:45.194524 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-ca109543-3110-4ab5-b7ee-947a3bf3e561 req-cb4c1d9e-2154-46fc-b682-e864a65f81f4 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030209 Jul 19 11:10:45.198390 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 487/952] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:10:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:10:45.247153 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-d313e3e6-b50c-4707-b35b-f9fb5ea258e0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:45.247766 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-d313e3e6-b50c-4707-b35b-f9fb5ea258e0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:45.254970 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d313e3e6-b50c-4707-b35b-f9fb5ea258e0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:45.255613 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d313e3e6-b50c-4707-b35b-f9fb5ea258e0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:45.256424 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d313e3e6-b50c-4707-b35b-f9fb5ea258e0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:45.257120 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d313e3e6-b50c-4707-b35b-f9fb5ea258e0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:45.406316 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-d313e3e6-b50c-4707-b35b-f9fb5ea258e0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:45.414543 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-d313e3e6-b50c-4707-b35b-f9fb5ea258e0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:45.416069 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-d313e3e6-b50c-4707-b35b-f9fb5ea258e0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:45.417268 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-d313e3e6-b50c-4707-b35b-f9fb5ea258e0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1723 microversion: 2.20 time: 0.174128 Jul 19 11:10:45.417824 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 466/953] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:10:45 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1723 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:10:45.615472 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-c89a6038-0192-4678-a07c-8adf00b57b28 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:45.616169 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-c89a6038-0192-4678-a07c-8adf00b57b28 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:45.625504 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c89a6038-0192-4678-a07c-8adf00b57b28 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:45.626150 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c89a6038-0192-4678-a07c-8adf00b57b28 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:45.626919 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c89a6038-0192-4678-a07c-8adf00b57b28 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:45.627352 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c89a6038-0192-4678-a07c-8adf00b57b28 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:45.695934 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-c89a6038-0192-4678-a07c-8adf00b57b28 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:45.704864 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-c89a6038-0192-4678-a07c-8adf00b57b28 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75" status: 200 len: 1290 microversion: 2.1 time: 0.100874 Jul 19 11:10:45.705450 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 488/954] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:45 2022] GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75 => generated 1290 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:46.055892 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-31624669-916b-42c2-9b45-df8da98ece9f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:46.057061 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-31624669-916b-42c2-9b45-df8da98ece9f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:46.064196 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-31624669-916b-42c2-9b45-df8da98ece9f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:46.064647 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-31624669-916b-42c2-9b45-df8da98ece9f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:46.065458 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-31624669-916b-42c2-9b45-df8da98ece9f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:46.065893 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-31624669-916b-42c2-9b45-df8da98ece9f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:46.069982 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=1,nova_cell1:INSERT=2 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:10:46.100405 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-31624669-916b-42c2-9b45-df8da98ece9f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:46.100890 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-31624669-916b-42c2-9b45-df8da98ece9f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:46.101348 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.objects.instance [None req-31624669-916b-42c2-9b45-df8da98ece9f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lazy-loading 'fault' on Instance uuid 03624642-ce02-450f-821d-1c135e5d7a4a {{(pid=117849) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 19 11:10:46.153779 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-31624669-916b-42c2-9b45-df8da98ece9f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e853b49c2c744e89243ec51e7a1a379', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '05630eeaf84a4d1bafd213d419f7a97a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:46.162500 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-31624669-916b-42c2-9b45-df8da98ece9f tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 200 len: 1348 microversion: 2.1 time: 0.111226 Jul 19 11:10:46.163075 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 467/955] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:46 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 1348 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:10:46.436250 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-579e2cfb-766a-4371-bb11-1c9d7e455801 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:46.437497 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-579e2cfb-766a-4371-bb11-1c9d7e455801 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:46.444857 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-579e2cfb-766a-4371-bb11-1c9d7e455801 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:46.445410 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-579e2cfb-766a-4371-bb11-1c9d7e455801 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:46.446286 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-579e2cfb-766a-4371-bb11-1c9d7e455801 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:46.446840 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-579e2cfb-766a-4371-bb11-1c9d7e455801 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:46.721458 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-93479e3d-ee02-4169-ba0a-356d63cb991a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:46.722061 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-93479e3d-ee02-4169-ba0a-356d63cb991a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:46.729265 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-93479e3d-ee02-4169-ba0a-356d63cb991a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:46.729719 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-93479e3d-ee02-4169-ba0a-356d63cb991a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:46.730622 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-93479e3d-ee02-4169-ba0a-356d63cb991a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:46.730923 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-93479e3d-ee02-4169-ba0a-356d63cb991a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:46.796552 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-93479e3d-ee02-4169-ba0a-356d63cb991a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:46.804094 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-93479e3d-ee02-4169-ba0a-356d63cb991a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75" status: 200 len: 1296 microversion: 2.1 time: 0.086398 Jul 19 11:10:46.805089 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 468/956] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:46 2022] GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75 => generated 1296 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:46.916594 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-579e2cfb-766a-4371-bb11-1c9d7e455801 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:46.926382 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-579e2cfb-766a-4371-bb11-1c9d7e455801 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:46.928051 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-579e2cfb-766a-4371-bb11-1c9d7e455801 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:46.929437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-579e2cfb-766a-4371-bb11-1c9d7e455801 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1723 microversion: 2.20 time: 0.498077 Jul 19 11:10:46.929994 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 489/957] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:10:46 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1723 bytes in 499 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:10:47.181525 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-6cde206a-f5d9-4f68-a327-f27d0bed24d5 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:47.182711 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-6cde206a-f5d9-4f68-a327-f27d0bed24d5 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] [instance: 03624642-ce02-450f-821d-1c135e5d7a4a] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:47.191221 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6cde206a-f5d9-4f68-a327-f27d0bed24d5 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:47.191456 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6cde206a-f5d9-4f68-a327-f27d0bed24d5 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:47.192454 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6cde206a-f5d9-4f68-a327-f27d0bed24d5 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:47.193033 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6cde206a-f5d9-4f68-a327-f27d0bed24d5 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:47.207569 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.wsgi [None req-6cde206a-f5d9-4f68-a327-f27d0bed24d5 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] HTTP exception thrown: Instance 03624642-ce02-450f-821d-1c135e5d7a4a could not be found. Jul 19 11:10:47.209460 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-6cde206a-f5d9-4f68-a327-f27d0bed24d5 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Request method failure captured: Jul 19 11:10:47.209460 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: request: GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a HTTP/1.1 Jul 19 11:10:47.209460 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept: application/json Jul 19 11:10:47.209460 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept-Encoding: identity Jul 19 11:10:47.209460 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Connection: close Jul 19 11:10:47.209460 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Length: 0 Jul 19 11:10:47.209460 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Type: application/json Jul 19 11:10:47.209460 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Host: 10.222.0.33 Jul 19 11:10:47.209460 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Openstack-System-Scope: None Jul 19 11:10:47.209460 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: User-Agent: python-urllib3/1.26.9 Jul 19 11:10:47.209460 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Auth-Token: gAAAAABi1pBqAQgKP5JyqcSTFiCeDPefAr26MUDPwxC3QXjPMYdp-MQklgz7HUR3fQSDcG9Si_QsMsJMAETbfQY7V9tN_aGZopZGDrOM9TTeR0tqssOhwFOYdDz6Te6fP1Zx6wlE_NJYQME2cqzGdLjiJX-9MS88De0sKIoUnxnUGcAJkGupjC4 Jul 19 11:10:47.209460 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Id: None Jul 19 11:10:47.209460 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Name: None Jul 19 11:10:47.209460 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Identity-Status: Confirmed Jul 19 11:10:47.209460 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Is-Admin-Project: True Jul 19 11:10:47.209460 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Id: default Jul 19 11:10:47.209460 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Name: Default Jul 19 11:10:47.209460 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Id: 05630eeaf84a4d1bafd213d419f7a97a Jul 19 11:10:47.209460 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Name: tempest-TestMinimumBasicScenario-1471164354 Jul 19 11:10:47.209460 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Role: member,reader Jul 19 11:10:47.209460 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Roles: member,reader Jul 19 11:10:47.210977 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/05630eeaf84a4d1bafd213d419f7a97a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/05630eeaf84a4d1bafd213d419f7a97a"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/05630eeaf84a4d1bafd213d419f7a97a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_05630eeaf84a4d1bafd213d419f7a97a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:10:47.210977 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant: tempest-TestMinimumBasicScenario-1471164354 Jul 19 11:10:47.210977 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Id: 05630eeaf84a4d1bafd213d419f7a97a Jul 19 11:10:47.210977 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1471164354 Jul 19 11:10:47.210977 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User: tempest-TestMinimumBasicScenario-1471164354-project Jul 19 11:10:47.210977 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Id: default Jul 19 11:10:47.210977 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Name: Default Jul 19 11:10:47.210977 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Id: 9e853b49c2c744e89243ec51e7a1a379 Jul 19 11:10:47.210977 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Name: tempest-TestMinimumBasicScenario-1471164354-project Jul 19 11:10:47.210977 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: method: > Jul 19 11:10:47.210977 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: exception: Instance 03624642-ce02-450f-821d-1c135e5d7a4a could not be found. Jul 19 11:10:47.210977 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: action_args: {'id': '03624642-ce02-450f-821d-1c135e5d7a4a'} Jul 19 11:10:47.211854 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:10:47.211854 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:47.211854 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:10:47.211854 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:10:47.211854 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:10:47.211854 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:10:47.211854 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:10:47.211854 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:10:47.211854 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:10:47.211854 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:10:47.211854 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 03624642-ce02-450f-821d-1c135e5d7a4a could not be found. Jul 19 11:10:47.211854 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:47.211854 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:10:47.211854 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:47.211854 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:47.211854 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:10:47.211854 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:10:47.211854 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:10:47.211854 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:10:47.211854 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:10:47.213246 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:10:47.213246 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 19 11:10:47.213246 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 19 11:10:47.213246 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:10:47.213246 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:10:47.213246 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:10:47.213246 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:10:47.213246 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 03624642-ce02-450f-821d-1c135e5d7a4a could not be found. Jul 19 11:10:47.213246 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:47.213246 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:10:47.213246 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:47.213246 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:10:47.213246 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:10:47.213246 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:10:47.213246 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:10:47.213246 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:10:47.213246 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 03624642-ce02-450f-821d-1c135e5d7a4a could not be found. Jul 19 11:10:47.213246 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:10:47.215248 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-6cde206a-f5d9-4f68-a327-f27d0bed24d5 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Returning 404 to user: Instance 03624642-ce02-450f-821d-1c135e5d7a4a could not be found. {{(pid=117849) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:10:47.215248 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-6cde206a-f5d9-4f68-a327-f27d0bed24d5 tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a" status: 404 len: 111 microversion: 2.1 time: 0.035307 Jul 19 11:10:47.215248 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 469/958] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:47 2022] GET /compute/v2.1/servers/03624642-ce02-450f-821d-1c135e5d7a4a => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:10:47.219487 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-a242895f-d7a9-429d-8635-3aae5f3cd5df tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] Calling method '.version_select at 0x7f04aa1f1670>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:47.237365 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-a242895f-d7a9-429d-8635-3aae5f3cd5df tempest-TestMinimumBasicScenario-1471164354 tempest-TestMinimumBasicScenario-1471164354-project] 10.222.0.33 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-468494935" status: 202 len: 0 microversion: 2.1 time: 0.021629 Jul 19 11:10:47.237950 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 490/959] 10.222.0.33 () {58 vars in 1268 bytes} [Tue Jul 19 11:10:47 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-468494935 => generated 0 bytes in 23 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 19 11:10:47.823147 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-38437c53-5379-41cf-974a-c55ec9110aff tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:47.824167 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-38437c53-5379-41cf-974a-c55ec9110aff tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:47.834293 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-38437c53-5379-41cf-974a-c55ec9110aff tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:47.834943 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-38437c53-5379-41cf-974a-c55ec9110aff tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:47.835567 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-38437c53-5379-41cf-974a-c55ec9110aff tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:47.836002 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-38437c53-5379-41cf-974a-c55ec9110aff tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:47.947423 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-60de4e47-10ec-49c7-aa00-de76c12749d8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:47.948018 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-60de4e47-10ec-49c7-aa00-de76c12749d8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:47.969437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-60de4e47-10ec-49c7-aa00-de76c12749d8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:47.970004 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-60de4e47-10ec-49c7-aa00-de76c12749d8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:47.970822 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-60de4e47-10ec-49c7-aa00-de76c12749d8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:47.971255 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-60de4e47-10ec-49c7-aa00-de76c12749d8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:48.333497 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-38437c53-5379-41cf-974a-c55ec9110aff tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:48.340949 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-38437c53-5379-41cf-974a-c55ec9110aff tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75" status: 200 len: 1363 microversion: 2.1 time: 0.522541 Jul 19 11:10:48.341518 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 470/960] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:47 2022] GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75 => generated 1363 bytes in 524 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:48.356487 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-60de4e47-10ec-49c7-aa00-de76c12749d8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:48.366124 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-60de4e47-10ec-49c7-aa00-de76c12749d8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:48.367713 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-60de4e47-10ec-49c7-aa00-de76c12749d8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:48.369057 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-60de4e47-10ec-49c7-aa00-de76c12749d8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1723 microversion: 2.20 time: 0.425438 Jul 19 11:10:48.369654 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 491/961] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:10:47 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1723 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:10:48.479434 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-3fb44456-6bea-4678-9c23-6afb43818b97 req-4b223450-fdb8-4427-9f81-0c7f798de27b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2c0cb519-166b-4f4d-ae3e-46309a6d7c75", "tag": "ce50b8aa-9d22-4ed1-84f5-50bee2bf7a72"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:10:48.491198 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-3fb44456-6bea-4678-9c23-6afb43818b97 req-4b223450-fdb8-4427-9f81-0c7f798de27b service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:48.491648 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-3fb44456-6bea-4678-9c23-6afb43818b97 req-4b223450-fdb8-4427-9f81-0c7f798de27b service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:48.505026 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-3fb44456-6bea-4678-9c23-6afb43818b97 req-4b223450-fdb8-4427-9f81-0c7f798de27b service nova] Creating event network-changed:ce50b8aa-9d22-4ed1-84f5-50bee2bf7a72 for instance 2c0cb519-166b-4f4d-ae3e-46309a6d7c75 on hv-cinder-maste Jul 19 11:10:48.510341 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-3fb44456-6bea-4678-9c23-6afb43818b97 req-4b223450-fdb8-4427-9f81-0c7f798de27b service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034580 Jul 19 11:10:48.527582 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 471/962] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:10:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:10:49.360557 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-b256229e-ff12-4ca0-9265-fde124c4f6f5 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:49.361214 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-b256229e-ff12-4ca0-9265-fde124c4f6f5 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:49.371076 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b256229e-ff12-4ca0-9265-fde124c4f6f5 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:49.371534 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b256229e-ff12-4ca0-9265-fde124c4f6f5 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:49.373598 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b256229e-ff12-4ca0-9265-fde124c4f6f5 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:49.373983 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b256229e-ff12-4ca0-9265-fde124c4f6f5 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:49.389626 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-880b4ff9-81bc-471c-8bea-37a8244b9d04 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:49.391009 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-880b4ff9-81bc-471c-8bea-37a8244b9d04 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:49.417234 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-880b4ff9-81bc-471c-8bea-37a8244b9d04 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:49.417806 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-880b4ff9-81bc-471c-8bea-37a8244b9d04 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:49.418593 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-880b4ff9-81bc-471c-8bea-37a8244b9d04 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:49.419038 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-880b4ff9-81bc-471c-8bea-37a8244b9d04 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:49.569183 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-880b4ff9-81bc-471c-8bea-37a8244b9d04 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:49.579525 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-880b4ff9-81bc-471c-8bea-37a8244b9d04 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:49.581081 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-880b4ff9-81bc-471c-8bea-37a8244b9d04 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:49.582242 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-880b4ff9-81bc-471c-8bea-37a8244b9d04 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1723 microversion: 2.20 time: 0.198262 Jul 19 11:10:49.582803 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 472/963] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:10:49 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1723 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:10:49.608500 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-b256229e-ff12-4ca0-9265-fde124c4f6f5 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:49.617961 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-b256229e-ff12-4ca0-9265-fde124c4f6f5 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75" status: 200 len: 1363 microversion: 2.1 time: 0.262589 Jul 19 11:10:49.618796 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 492/964] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:49 2022] GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75 => generated 1363 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:50.601247 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-53fc4126-a23b-4ba1-8fe1-22ed195ebcc5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:50.602514 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-53fc4126-a23b-4ba1-8fe1-22ed195ebcc5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:50.610224 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-53fc4126-a23b-4ba1-8fe1-22ed195ebcc5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:50.611075 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-53fc4126-a23b-4ba1-8fe1-22ed195ebcc5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:50.611854 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-53fc4126-a23b-4ba1-8fe1-22ed195ebcc5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:50.612444 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-53fc4126-a23b-4ba1-8fe1-22ed195ebcc5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:50.635631 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-21c2e0cb-57a7-4f19-9413-d59b37bd795a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:50.636258 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-21c2e0cb-57a7-4f19-9413-d59b37bd795a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:50.649240 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-21c2e0cb-57a7-4f19-9413-d59b37bd795a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:50.649698 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-21c2e0cb-57a7-4f19-9413-d59b37bd795a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:50.650512 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-21c2e0cb-57a7-4f19-9413-d59b37bd795a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:50.650952 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-21c2e0cb-57a7-4f19-9413-d59b37bd795a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:50.888780 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-21c2e0cb-57a7-4f19-9413-d59b37bd795a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:50.902071 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-21c2e0cb-57a7-4f19-9413-d59b37bd795a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75" status: 200 len: 1363 microversion: 2.1 time: 0.270222 Jul 19 11:10:50.903386 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 493/965] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:50 2022] GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75 => generated 1363 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:50.979159 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-53fc4126-a23b-4ba1-8fe1-22ed195ebcc5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:50.988458 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-53fc4126-a23b-4ba1-8fe1-22ed195ebcc5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:50.990414 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-53fc4126-a23b-4ba1-8fe1-22ed195ebcc5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:51.003669 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-53fc4126-a23b-4ba1-8fe1-22ed195ebcc5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1723 microversion: 2.20 time: 0.406288 Jul 19 11:10:51.004520 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 473/966] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:10:50 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1723 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:10:51.927009 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-c55b9904-1825-4879-84da-0996ad0045ff tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:51.938354 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-c55b9904-1825-4879-84da-0996ad0045ff tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:51.949143 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c55b9904-1825-4879-84da-0996ad0045ff tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:51.949603 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c55b9904-1825-4879-84da-0996ad0045ff tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:51.950391 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c55b9904-1825-4879-84da-0996ad0045ff tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:51.950834 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c55b9904-1825-4879-84da-0996ad0045ff tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:52.022472 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-d6ff1e7f-2270-4379-9e2f-96c635780992 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:52.023095 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-d6ff1e7f-2270-4379-9e2f-96c635780992 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:52.030743 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d6ff1e7f-2270-4379-9e2f-96c635780992 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:52.031195 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d6ff1e7f-2270-4379-9e2f-96c635780992 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:52.034189 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d6ff1e7f-2270-4379-9e2f-96c635780992 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:52.036419 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d6ff1e7f-2270-4379-9e2f-96c635780992 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:52.131219 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-c55b9904-1825-4879-84da-0996ad0045ff tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:52.138982 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-c55b9904-1825-4879-84da-0996ad0045ff tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75" status: 200 len: 1363 microversion: 2.1 time: 0.221282 Jul 19 11:10:52.148323 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 494/967] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:51 2022] GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75 => generated 1363 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:52.285552 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-d6ff1e7f-2270-4379-9e2f-96c635780992 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:52.295341 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-d6ff1e7f-2270-4379-9e2f-96c635780992 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:52.297013 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-d6ff1e7f-2270-4379-9e2f-96c635780992 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:52.298187 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-d6ff1e7f-2270-4379-9e2f-96c635780992 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1744 microversion: 2.20 time: 0.279841 Jul 19 11:10:52.299069 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 474/968] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:10:52 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1744 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:10:53.156497 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-331140e7-20f7-4604-aa7c-78a8fc9cee90 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:53.157173 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-331140e7-20f7-4604-aa7c-78a8fc9cee90 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:53.163970 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-331140e7-20f7-4604-aa7c-78a8fc9cee90 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:53.164453 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-331140e7-20f7-4604-aa7c-78a8fc9cee90 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:53.165310 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-331140e7-20f7-4604-aa7c-78a8fc9cee90 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:53.165785 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-331140e7-20f7-4604-aa7c-78a8fc9cee90 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:53.300927 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-331140e7-20f7-4604-aa7c-78a8fc9cee90 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:53.308463 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-331140e7-20f7-4604-aa7c-78a8fc9cee90 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75" status: 200 len: 1363 microversion: 2.1 time: 0.156339 Jul 19 11:10:53.310867 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 495/969] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:53 2022] GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75 => generated 1363 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:53.315280 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-895140fe-5226-40e7-b59c-793cb8f7013e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:53.316394 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-895140fe-5226-40e7-b59c-793cb8f7013e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:53.322888 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-895140fe-5226-40e7-b59c-793cb8f7013e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:53.323342 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-895140fe-5226-40e7-b59c-793cb8f7013e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:53.324110 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-895140fe-5226-40e7-b59c-793cb8f7013e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:53.324542 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-895140fe-5226-40e7-b59c-793cb8f7013e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:53.505186 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-895140fe-5226-40e7-b59c-793cb8f7013e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:53.513043 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-895140fe-5226-40e7-b59c-793cb8f7013e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:53.515223 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-895140fe-5226-40e7-b59c-793cb8f7013e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:53.516386 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-895140fe-5226-40e7-b59c-793cb8f7013e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1744 microversion: 2.20 time: 0.204875 Jul 19 11:10:53.517165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 475/970] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:10:53 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1744 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:10:54.326476 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-4d1f35d7-43c9-4427-b67a-01fd61d2bcf3 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:54.327066 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-4d1f35d7-43c9-4427-b67a-01fd61d2bcf3 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:54.333674 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4d1f35d7-43c9-4427-b67a-01fd61d2bcf3 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:54.334126 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4d1f35d7-43c9-4427-b67a-01fd61d2bcf3 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:54.334895 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4d1f35d7-43c9-4427-b67a-01fd61d2bcf3 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:54.335330 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4d1f35d7-43c9-4427-b67a-01fd61d2bcf3 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:54.472381 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-4d1f35d7-43c9-4427-b67a-01fd61d2bcf3 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:54.480805 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-4d1f35d7-43c9-4427-b67a-01fd61d2bcf3 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75" status: 200 len: 1363 microversion: 2.1 time: 0.157841 Jul 19 11:10:54.481931 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 496/971] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:54 2022] GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75 => generated 1363 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:54.535809 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-424ec788-ee7f-41f7-8551-609cf5646e6a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:54.537184 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-424ec788-ee7f-41f7-8551-609cf5646e6a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:54.542850 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell0:SELECT=2 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:10:54.543489 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell0:SELECT=6 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:10:54.546274 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-424ec788-ee7f-41f7-8551-609cf5646e6a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:54.546831 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-424ec788-ee7f-41f7-8551-609cf5646e6a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:54.547596 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-424ec788-ee7f-41f7-8551-609cf5646e6a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:54.548027 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-424ec788-ee7f-41f7-8551-609cf5646e6a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:54.747343 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-424ec788-ee7f-41f7-8551-609cf5646e6a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:54.760196 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-424ec788-ee7f-41f7-8551-609cf5646e6a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:54.762527 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-424ec788-ee7f-41f7-8551-609cf5646e6a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:54.764107 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-424ec788-ee7f-41f7-8551-609cf5646e6a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1744 microversion: 2.20 time: 0.231819 Jul 19 11:10:54.765010 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 476/972] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:10:54 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1744 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:10:55.504898 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-3106116d-28b4-48db-b043-0d5f865c5fb0 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:55.505655 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-3106116d-28b4-48db-b043-0d5f865c5fb0 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:55.508811 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell0:SELECT=6 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:10:55.519646 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3106116d-28b4-48db-b043-0d5f865c5fb0 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:55.521044 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3106116d-28b4-48db-b043-0d5f865c5fb0 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:55.522991 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3106116d-28b4-48db-b043-0d5f865c5fb0 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:55.523684 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3106116d-28b4-48db-b043-0d5f865c5fb0 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:55.782673 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-7bfec0d9-ef87-4698-9852-122d804253ea tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:55.783947 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-7bfec0d9-ef87-4698-9852-122d804253ea tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:55.800426 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7bfec0d9-ef87-4698-9852-122d804253ea tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:55.807983 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7bfec0d9-ef87-4698-9852-122d804253ea tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:55.809613 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7bfec0d9-ef87-4698-9852-122d804253ea tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:55.809918 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7bfec0d9-ef87-4698-9852-122d804253ea tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:56.016130 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-3106116d-28b4-48db-b043-0d5f865c5fb0 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:56.037444 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-3106116d-28b4-48db-b043-0d5f865c5fb0 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75" status: 200 len: 1363 microversion: 2.1 time: 0.541082 Jul 19 11:10:56.042585 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 497/973] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:55 2022] GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75 => generated 1363 bytes in 548 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:56.098808 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-7bfec0d9-ef87-4698-9852-122d804253ea tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:56.106817 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-7bfec0d9-ef87-4698-9852-122d804253ea tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:56.125004 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-7bfec0d9-ef87-4698-9852-122d804253ea tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:56.126912 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-7bfec0d9-ef87-4698-9852-122d804253ea tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1739 microversion: 2.20 time: 0.348742 Jul 19 11:10:56.127957 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 477/974] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:10:55 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1739 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:10:56.962899 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-1f0bb07b-1ded-4ce1-b794-357b4008a3d0 req-7386a724-b6dc-4a40-a919-4c55298035ea service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2c0cb519-166b-4f4d-ae3e-46309a6d7c75", "name": "network-vif-plugged", "status": "completed", "tag": "ce50b8aa-9d22-4ed1-84f5-50bee2bf7a72"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:10:56.976747 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-1f0bb07b-1ded-4ce1-b794-357b4008a3d0 req-7386a724-b6dc-4a40-a919-4c55298035ea service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:56.977187 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-1f0bb07b-1ded-4ce1-b794-357b4008a3d0 req-7386a724-b6dc-4a40-a919-4c55298035ea service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:56.982455 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:10:57.002060 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-1f0bb07b-1ded-4ce1-b794-357b4008a3d0 req-7386a724-b6dc-4a40-a919-4c55298035ea service nova] Creating event network-vif-plugged:ce50b8aa-9d22-4ed1-84f5-50bee2bf7a72 for instance 2c0cb519-166b-4f4d-ae3e-46309a6d7c75 on hv-cinder-maste Jul 19 11:10:57.026558 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-1f0bb07b-1ded-4ce1-b794-357b4008a3d0 req-7386a724-b6dc-4a40-a919-4c55298035ea service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.067236 Jul 19 11:10:57.027172 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 498/975] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:10:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:10:57.057058 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-0e46ab70-6d09-492b-b874-0ead41c7ee3f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:57.057881 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-0e46ab70-6d09-492b-b874-0ead41c7ee3f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:57.073114 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0e46ab70-6d09-492b-b874-0ead41c7ee3f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:57.083621 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0e46ab70-6d09-492b-b874-0ead41c7ee3f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:57.086185 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0e46ab70-6d09-492b-b874-0ead41c7ee3f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:57.090925 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0e46ab70-6d09-492b-b874-0ead41c7ee3f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:57.152911 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-e748a5f8-c134-490c-b8f6-b38ebdcd18b0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:57.154295 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-e748a5f8-c134-490c-b8f6-b38ebdcd18b0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:57.167711 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e748a5f8-c134-490c-b8f6-b38ebdcd18b0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:57.178579 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e748a5f8-c134-490c-b8f6-b38ebdcd18b0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:57.183912 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e748a5f8-c134-490c-b8f6-b38ebdcd18b0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:57.184934 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e748a5f8-c134-490c-b8f6-b38ebdcd18b0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:57.349018 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-0e46ab70-6d09-492b-b874-0ead41c7ee3f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:57.356680 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-0e46ab70-6d09-492b-b874-0ead41c7ee3f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75" status: 200 len: 1363 microversion: 2.1 time: 0.304889 Jul 19 11:10:57.363776 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 478/976] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:57 2022] GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75 => generated 1363 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:57.533152 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-e748a5f8-c134-490c-b8f6-b38ebdcd18b0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:57.543643 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-e748a5f8-c134-490c-b8f6-b38ebdcd18b0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:57.546989 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-e748a5f8-c134-490c-b8f6-b38ebdcd18b0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:57.549162 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-e748a5f8-c134-490c-b8f6-b38ebdcd18b0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1739 microversion: 2.20 time: 0.407029 Jul 19 11:10:57.551302 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 499/977] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:10:57 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1739 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:10:58.374947 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-725cd64b-f269-4d40-9b96-ac4266f7628f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:58.379311 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-725cd64b-f269-4d40-9b96-ac4266f7628f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:58.388037 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-725cd64b-f269-4d40-9b96-ac4266f7628f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:58.389355 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-725cd64b-f269-4d40-9b96-ac4266f7628f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:58.390402 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-725cd64b-f269-4d40-9b96-ac4266f7628f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:58.391064 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-725cd64b-f269-4d40-9b96-ac4266f7628f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:58.567360 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-3417dfbb-e808-4545-a050-5c7a8c63c90b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:58.567971 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-3417dfbb-e808-4545-a050-5c7a8c63c90b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:58.581228 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3417dfbb-e808-4545-a050-5c7a8c63c90b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:58.581958 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3417dfbb-e808-4545-a050-5c7a8c63c90b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:58.582924 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3417dfbb-e808-4545-a050-5c7a8c63c90b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:58.588447 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-725cd64b-f269-4d40-9b96-ac4266f7628f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:58.593484 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3417dfbb-e808-4545-a050-5c7a8c63c90b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:58.597779 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-725cd64b-f269-4d40-9b96-ac4266f7628f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75" status: 200 len: 1363 microversion: 2.1 time: 0.227538 Jul 19 11:10:58.601525 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 479/978] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:58 2022] GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75 => generated 1363 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:58.728877 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-dd416be5-558e-4e56-b11e-41a9739347b1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:58.729381 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-dd416be5-558e-4e56-b11e-41a9739347b1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 12cd8c55-b30f-40ee-8cf5-7c974b8caafb] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:58.740485 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dd416be5-558e-4e56-b11e-41a9739347b1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:58.740944 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dd416be5-558e-4e56-b11e-41a9739347b1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:58.741787 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dd416be5-558e-4e56-b11e-41a9739347b1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:58.742232 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dd416be5-558e-4e56-b11e-41a9739347b1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:58.766716 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-3417dfbb-e808-4545-a050-5c7a8c63c90b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:58.777179 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-3417dfbb-e808-4545-a050-5c7a8c63c90b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:58.781221 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-3417dfbb-e808-4545-a050-5c7a8c63c90b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:58.782701 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-3417dfbb-e808-4545-a050-5c7a8c63c90b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1739 microversion: 2.20 time: 0.219218 Jul 19 11:10:58.786998 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 500/979] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:10:58 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1739 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:10:58.801701 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-dd416be5-558e-4e56-b11e-41a9739347b1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 12cd8c55-b30f-40ee-8cf5-7c974b8caafb] Going to try to terminate instance {{(pid=117849) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 19 11:10:58.900162 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-dd416be5-558e-4e56-b11e-41a9739347b1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "DELETE /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb" status: 204 len: 0 microversion: 2.1 time: 0.174876 Jul 19 11:10:58.901470 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 480/980] 10.222.0.33 () {58 vars in 1238 bytes} [Tue Jul 19 11:10:58 2022] DELETE /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb => generated 0 bytes in 177 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 19 11:10:58.910851 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-89e84311-a51e-4f60-8eae-9db3980025e1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:58.911639 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-89e84311-a51e-4f60-8eae-9db3980025e1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 12cd8c55-b30f-40ee-8cf5-7c974b8caafb] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:58.926208 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-89e84311-a51e-4f60-8eae-9db3980025e1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:58.926986 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-89e84311-a51e-4f60-8eae-9db3980025e1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:58.927981 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-89e84311-a51e-4f60-8eae-9db3980025e1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:58.928607 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-89e84311-a51e-4f60-8eae-9db3980025e1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:59.053052 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-bed89df2-4869-4560-8bd3-1a3832e3aa52 req-f9131a20-5a4d-473b-ad74-291647f46b99 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "36abf2ae-3b4c-4095-b9ba-2bd738e266d9", "name": "network-vif-plugged", "status": "completed", "tag": "049ef62d-5f12-4134-9010-2bc20edecd41"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:10:59.063903 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-bed89df2-4869-4560-8bd3-1a3832e3aa52 req-f9131a20-5a4d-473b-ad74-291647f46b99 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:59.064357 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-bed89df2-4869-4560-8bd3-1a3832e3aa52 req-f9131a20-5a4d-473b-ad74-291647f46b99 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:59.085626 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-89e84311-a51e-4f60-8eae-9db3980025e1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:59.088173 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-bed89df2-4869-4560-8bd3-1a3832e3aa52 req-f9131a20-5a4d-473b-ad74-291647f46b99 service nova] Creating event network-vif-plugged:049ef62d-5f12-4134-9010-2bc20edecd41 for instance 36abf2ae-3b4c-4095-b9ba-2bd738e266d9 on hv-cinder-maste Jul 19 11:10:59.095574 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-89e84311-a51e-4f60-8eae-9db3980025e1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb" status: 200 len: 1577 microversion: 2.1 time: 0.188807 Jul 19 11:10:59.095859 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-bed89df2-4869-4560-8bd3-1a3832e3aa52 req-f9131a20-5a4d-473b-ad74-291647f46b99 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.047043 Jul 19 11:10:59.096379 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 501/981] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:58 2022] GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb => generated 1577 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:59.100821 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 481/982] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:10:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:10:59.618197 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-547ed055-fcaa-49e0-a3b3-aacfef29f3eb tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:59.619386 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-547ed055-fcaa-49e0-a3b3-aacfef29f3eb tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:59.626633 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-547ed055-fcaa-49e0-a3b3-aacfef29f3eb tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:59.627357 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-547ed055-fcaa-49e0-a3b3-aacfef29f3eb tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:59.628361 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-547ed055-fcaa-49e0-a3b3-aacfef29f3eb tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:59.629071 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-547ed055-fcaa-49e0-a3b3-aacfef29f3eb tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:59.804718 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-b55951f2-6760-4e3f-b5e8-6b1d3288292f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:59.806183 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-b55951f2-6760-4e3f-b5e8-6b1d3288292f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:59.808257 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-547ed055-fcaa-49e0-a3b3-aacfef29f3eb tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:10:59.817765 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b55951f2-6760-4e3f-b5e8-6b1d3288292f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:59.818665 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b55951f2-6760-4e3f-b5e8-6b1d3288292f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:59.821232 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b55951f2-6760-4e3f-b5e8-6b1d3288292f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:59.822660 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b55951f2-6760-4e3f-b5e8-6b1d3288292f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:59.829334 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-547ed055-fcaa-49e0-a3b3-aacfef29f3eb tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75" status: 200 len: 1550 microversion: 2.1 time: 0.214887 Jul 19 11:10:59.832500 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 502/983] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:10:59 2022] GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75 => generated 1550 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:10:59.852466 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-301e8340-d066-4264-8f24-4d2317346499 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:10:59.853215 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-301e8340-d066-4264-8f24-4d2317346499 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:10:59.861317 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-301e8340-d066-4264-8f24-4d2317346499 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:59.862054 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-301e8340-d066-4264-8f24-4d2317346499 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:59.864546 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-301e8340-d066-4264-8f24-4d2317346499 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:10:59.865215 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-301e8340-d066-4264-8f24-4d2317346499 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:10:59.994902 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-301e8340-d066-4264-8f24-4d2317346499 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.146716 Jul 19 11:10:59.996201 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 503/984] 10.222.0.33 () {58 vars in 1274 bytes} [Tue Jul 19 11:10:59 2022] GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75/os-interface => generated 285 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:11:00.064554 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b55951f2-6760-4e3f-b5e8-6b1d3288292f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:00.072733 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b55951f2-6760-4e3f-b5e8-6b1d3288292f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:00.074265 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b55951f2-6760-4e3f-b5e8-6b1d3288292f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:00.075418 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-b55951f2-6760-4e3f-b5e8-6b1d3288292f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1739 microversion: 2.20 time: 0.275801 Jul 19 11:11:00.076022 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 482/985] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:10:59 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1739 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:11:00.114720 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-aca61065-e6d2-41cc-8fc5-ff14174e6dcd tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:00.115982 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-aca61065-e6d2-41cc-8fc5-ff14174e6dcd tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 12cd8c55-b30f-40ee-8cf5-7c974b8caafb] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:00.127712 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-aca61065-e6d2-41cc-8fc5-ff14174e6dcd tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:00.128165 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-aca61065-e6d2-41cc-8fc5-ff14174e6dcd tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:00.132158 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-aca61065-e6d2-41cc-8fc5-ff14174e6dcd tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:00.132624 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-aca61065-e6d2-41cc-8fc5-ff14174e6dcd tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:00.399947 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-aca61065-e6d2-41cc-8fc5-ff14174e6dcd tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:00.413979 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-aca61065-e6d2-41cc-8fc5-ff14174e6dcd tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb" status: 200 len: 1577 microversion: 2.1 time: 0.303475 Jul 19 11:11:00.415129 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 504/986] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:00 2022] GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb => generated 1577 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:01.094773 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-45df842e-4be3-48bb-9b90-85693264e4e4 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:01.096036 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-45df842e-4be3-48bb-9b90-85693264e4e4 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:01.113678 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-45df842e-4be3-48bb-9b90-85693264e4e4 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:01.114351 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-45df842e-4be3-48bb-9b90-85693264e4e4 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:01.115321 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-45df842e-4be3-48bb-9b90-85693264e4e4 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:01.115926 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-45df842e-4be3-48bb-9b90-85693264e4e4 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:01.275780 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-45df842e-4be3-48bb-9b90-85693264e4e4 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:01.280467 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:11:01.292309 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-45df842e-4be3-48bb-9b90-85693264e4e4 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:01.294146 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-45df842e-4be3-48bb-9b90-85693264e4e4 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:01.295971 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-45df842e-4be3-48bb-9b90-85693264e4e4 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1739 microversion: 2.20 time: 0.205638 Jul 19 11:11:01.301191 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 483/987] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:01 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1739 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:01.436705 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-3ccf8528-ef3a-448d-bae6-41695e333152 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:01.439164 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-3ccf8528-ef3a-448d-bae6-41695e333152 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 12cd8c55-b30f-40ee-8cf5-7c974b8caafb] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:01.450179 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3ccf8528-ef3a-448d-bae6-41695e333152 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:01.450630 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3ccf8528-ef3a-448d-bae6-41695e333152 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:01.453571 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3ccf8528-ef3a-448d-bae6-41695e333152 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:01.455460 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3ccf8528-ef3a-448d-bae6-41695e333152 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:01.517287 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-090dd26c-22ce-40af-901e-ddd241d71b0e req-4191ec49-ba76-4a1c-8e9e-d7593a2dc3cb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2c0cb519-166b-4f4d-ae3e-46309a6d7c75", "tag": "ce50b8aa-9d22-4ed1-84f5-50bee2bf7a72"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:11:01.533293 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-090dd26c-22ce-40af-901e-ddd241d71b0e req-4191ec49-ba76-4a1c-8e9e-d7593a2dc3cb service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:01.533981 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-090dd26c-22ce-40af-901e-ddd241d71b0e req-4191ec49-ba76-4a1c-8e9e-d7593a2dc3cb service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:01.561559 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-090dd26c-22ce-40af-901e-ddd241d71b0e req-4191ec49-ba76-4a1c-8e9e-d7593a2dc3cb service nova] Creating event network-changed:ce50b8aa-9d22-4ed1-84f5-50bee2bf7a72 for instance 2c0cb519-166b-4f4d-ae3e-46309a6d7c75 on hv-cinder-maste Jul 19 11:11:01.572650 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-090dd26c-22ce-40af-901e-ddd241d71b0e req-4191ec49-ba76-4a1c-8e9e-d7593a2dc3cb service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.059981 Jul 19 11:11:01.577371 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 484/988] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:11:01 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:11:01.869323 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-3ccf8528-ef3a-448d-bae6-41695e333152 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:01.883107 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-3ccf8528-ef3a-448d-bae6-41695e333152 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb" status: 200 len: 1577 microversion: 2.1 time: 0.455191 Jul 19 11:11:01.886014 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 505/989] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:01 2022] GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb => generated 1577 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:02.317383 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-cdb8a16b-f755-4a4d-867c-5b00beb3aef0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:02.318084 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-cdb8a16b-f755-4a4d-867c-5b00beb3aef0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:02.328054 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cdb8a16b-f755-4a4d-867c-5b00beb3aef0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:02.328509 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cdb8a16b-f755-4a4d-867c-5b00beb3aef0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:02.329317 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cdb8a16b-f755-4a4d-867c-5b00beb3aef0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:02.329758 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cdb8a16b-f755-4a4d-867c-5b00beb3aef0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:02.733910 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-cdb8a16b-f755-4a4d-867c-5b00beb3aef0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:02.767462 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-cdb8a16b-f755-4a4d-867c-5b00beb3aef0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:02.772147 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-cdb8a16b-f755-4a4d-867c-5b00beb3aef0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:02.782097 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-cdb8a16b-f755-4a4d-867c-5b00beb3aef0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1739 microversion: 2.20 time: 0.462667 Jul 19 11:11:02.790179 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 485/990] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:02 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1739 bytes in 480 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:02.904653 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-21a29ee5-8885-4458-8610-21283d75191f tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:02.906949 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-21a29ee5-8885-4458-8610-21283d75191f tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 12cd8c55-b30f-40ee-8cf5-7c974b8caafb] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:02.925230 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-21a29ee5-8885-4458-8610-21283d75191f tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:02.927553 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-21a29ee5-8885-4458-8610-21283d75191f tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:02.929787 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-21a29ee5-8885-4458-8610-21283d75191f tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:02.930241 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-21a29ee5-8885-4458-8610-21283d75191f tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:03.078110 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-21a29ee5-8885-4458-8610-21283d75191f tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:03.090402 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-21a29ee5-8885-4458-8610-21283d75191f tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb" status: 200 len: 1225 microversion: 2.1 time: 0.192370 Jul 19 11:11:03.090996 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 506/991] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:02 2022] GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb => generated 1225 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:03.643374 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-c720e1e3-9042-4bd3-80fe-487d076bd351 req-17d4b482-bd6a-4474-98d1-bb6104523252 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "12cd8c55-b30f-40ee-8cf5-7c974b8caafb", "name": "network-vif-deleted", "tag": "2648a82a-50bf-410f-86dd-82693a421316"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:11:03.673019 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-c720e1e3-9042-4bd3-80fe-487d076bd351 req-17d4b482-bd6a-4474-98d1-bb6104523252 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:03.673890 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-c720e1e3-9042-4bd3-80fe-487d076bd351 req-17d4b482-bd6a-4474-98d1-bb6104523252 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:03.708308 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-c720e1e3-9042-4bd3-80fe-487d076bd351 req-17d4b482-bd6a-4474-98d1-bb6104523252 service nova] Creating event network-vif-deleted:2648a82a-50bf-410f-86dd-82693a421316 for instance 12cd8c55-b30f-40ee-8cf5-7c974b8caafb on hv-cinder-maste Jul 19 11:11:03.723306 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-c720e1e3-9042-4bd3-80fe-487d076bd351 req-17d4b482-bd6a-4474-98d1-bb6104523252 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.074321 Jul 19 11:11:03.733992 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 486/992] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:11:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:11:03.801751 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-60dae891-06ee-446f-83c1-36ec1a7851ca tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:03.802609 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-60dae891-06ee-446f-83c1-36ec1a7851ca tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:03.809493 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-60dae891-06ee-446f-83c1-36ec1a7851ca tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:03.810826 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-60dae891-06ee-446f-83c1-36ec1a7851ca tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:03.811834 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-60dae891-06ee-446f-83c1-36ec1a7851ca tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:03.824951 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-60dae891-06ee-446f-83c1-36ec1a7851ca tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:04.030648 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-60dae891-06ee-446f-83c1-36ec1a7851ca tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:04.044250 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-60dae891-06ee-446f-83c1-36ec1a7851ca tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:04.045828 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-60dae891-06ee-446f-83c1-36ec1a7851ca tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:04.047183 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-60dae891-06ee-446f-83c1-36ec1a7851ca tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1739 microversion: 2.20 time: 0.249810 Jul 19 11:11:04.047951 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 507/993] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:03 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1739 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:04.108627 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-074e2677-d0fc-4e67-8a13-75feae755a34 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:04.109414 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-074e2677-d0fc-4e67-8a13-75feae755a34 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 12cd8c55-b30f-40ee-8cf5-7c974b8caafb] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:04.116041 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-074e2677-d0fc-4e67-8a13-75feae755a34 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:04.116495 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-074e2677-d0fc-4e67-8a13-75feae755a34 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:04.117285 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-074e2677-d0fc-4e67-8a13-75feae755a34 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:04.117795 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-074e2677-d0fc-4e67-8a13-75feae755a34 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:04.137288 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.wsgi [None req-074e2677-d0fc-4e67-8a13-75feae755a34 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] HTTP exception thrown: Instance 12cd8c55-b30f-40ee-8cf5-7c974b8caafb could not be found. Jul 19 11:11:04.139432 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-074e2677-d0fc-4e67-8a13-75feae755a34 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Request method failure captured: Jul 19 11:11:04.139432 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: request: GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb HTTP/1.1 Jul 19 11:11:04.139432 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept: application/json Jul 19 11:11:04.139432 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept-Encoding: identity Jul 19 11:11:04.139432 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Connection: close Jul 19 11:11:04.139432 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Length: 0 Jul 19 11:11:04.139432 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Type: application/json Jul 19 11:11:04.139432 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Host: 10.222.0.33 Jul 19 11:11:04.139432 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Openstack-System-Scope: None Jul 19 11:11:04.139432 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: User-Agent: python-urllib3/1.26.9 Jul 19 11:11:04.139432 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Auth-Token: gAAAAABi1pCKSZ-4SJ5s_ItAgtEK2gvn4DyrOW7x3YHwCqO5aRGl0Sxm4ZZRq5InRFnucVfPkLXEpzi5hBz6xHVKVr9nFez8ydrjVvbO97mMRi8k_yB3xxgwAy6BC0nPq6dyYILo3FmB20Si0ydiOAi4CYj6_z7hgsm3RTs6jezifIIUuhEeGdE Jul 19 11:11:04.139432 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Id: None Jul 19 11:11:04.139432 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Name: None Jul 19 11:11:04.139432 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Identity-Status: Confirmed Jul 19 11:11:04.139432 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Is-Admin-Project: True Jul 19 11:11:04.139432 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Id: default Jul 19 11:11:04.139432 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Name: Default Jul 19 11:11:04.139432 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Id: a7109adf88ac4391aee35c150e96e0b1 Jul 19 11:11:04.139432 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Name: tempest-TestVolumeBootPattern-1429943467 Jul 19 11:11:04.139432 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Role: member,reader Jul 19 11:11:04.139432 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Roles: member,reader Jul 19 11:11:04.140810 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7109adf88ac4391aee35c150e96e0b1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/a7109adf88ac4391aee35c150e96e0b1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7109adf88ac4391aee35c150e96e0b1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_a7109adf88ac4391aee35c150e96e0b1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:11:04.140810 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant: tempest-TestVolumeBootPattern-1429943467 Jul 19 11:11:04.140810 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Id: a7109adf88ac4391aee35c150e96e0b1 Jul 19 11:11:04.140810 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Name: tempest-TestVolumeBootPattern-1429943467 Jul 19 11:11:04.140810 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User: tempest-TestVolumeBootPattern-1429943467-project Jul 19 11:11:04.140810 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Id: default Jul 19 11:11:04.140810 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Name: Default Jul 19 11:11:04.140810 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Id: a217e983effa43f3a4f84221984189c5 Jul 19 11:11:04.140810 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Name: tempest-TestVolumeBootPattern-1429943467-project Jul 19 11:11:04.140810 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: method: > Jul 19 11:11:04.140810 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: exception: Instance 12cd8c55-b30f-40ee-8cf5-7c974b8caafb could not be found. Jul 19 11:11:04.140810 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: action_args: {'id': '12cd8c55-b30f-40ee-8cf5-7c974b8caafb'} Jul 19 11:11:04.141596 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:11:04.141596 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:11:04.141596 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:11:04.141596 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:11:04.141596 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:11:04.141596 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:11:04.141596 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:11:04.141596 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:11:04.141596 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:11:04.141596 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:11:04.141596 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 12cd8c55-b30f-40ee-8cf5-7c974b8caafb could not be found. Jul 19 11:11:04.141596 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:11:04.141596 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:11:04.141596 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:11:04.141596 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:11:04.141596 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:11:04.141596 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:11:04.141596 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:11:04.141596 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:11:04.141596 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:11:04.142855 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:11:04.142855 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 19 11:11:04.142855 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 19 11:11:04.142855 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:11:04.142855 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:11:04.142855 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:11:04.142855 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:11:04.142855 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 12cd8c55-b30f-40ee-8cf5-7c974b8caafb could not be found. Jul 19 11:11:04.142855 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:11:04.142855 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:11:04.142855 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:11:04.142855 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:11:04.142855 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:11:04.142855 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:11:04.142855 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:11:04.142855 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:11:04.142855 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 12cd8c55-b30f-40ee-8cf5-7c974b8caafb could not be found. Jul 19 11:11:04.142855 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:11:04.147191 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-074e2677-d0fc-4e67-8a13-75feae755a34 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Returning 404 to user: Instance 12cd8c55-b30f-40ee-8cf5-7c974b8caafb could not be found. {{(pid=117849) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:11:04.147191 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-074e2677-d0fc-4e67-8a13-75feae755a34 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb" status: 404 len: 111 microversion: 2.1 time: 0.037179 Jul 19 11:11:04.147191 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 487/994] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:04 2022] GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:11:04.262669 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-cc8644fb-720b-4dfa-a2a7-3442554dfeaf tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1183995569", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1205270003", "security_groups": [{"name": "tempest-secgroup-smoke-1334719817"}], "block_device_mapping_v2": [{"uuid": "2125ea25-7fd0-42f0-ace7-b5ad9ed45ebd", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "1660664f-e111-4b9d-93da-a8c613b3e4df"}]}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:11:04.449126 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.network.neutron [None req-cc8644fb-720b-4dfa-a2a7-3442554dfeaf tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] validate_networks() for [('1660664f-e111-4b9d-93da-a8c613b3e4df', None, None, None, None, None)] {{(pid=117848) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 19 11:11:05.069835 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-cb589a01-b50f-4f90-9cc4-06196aabfcad tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:05.072713 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-cb589a01-b50f-4f90-9cc4-06196aabfcad tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:05.085953 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cb589a01-b50f-4f90-9cc4-06196aabfcad tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:05.092435 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cb589a01-b50f-4f90-9cc4-06196aabfcad tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:05.095586 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cb589a01-b50f-4f90-9cc4-06196aabfcad tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:05.096385 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cb589a01-b50f-4f90-9cc4-06196aabfcad tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:05.299408 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-cb589a01-b50f-4f90-9cc4-06196aabfcad tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:05.314970 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-cb589a01-b50f-4f90-9cc4-06196aabfcad tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:05.325537 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-cb589a01-b50f-4f90-9cc4-06196aabfcad tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:05.334310 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-cb589a01-b50f-4f90-9cc4-06196aabfcad tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1739 microversion: 2.20 time: 0.272132 Jul 19 11:11:05.334737 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 488/995] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:05 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1739 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:05.345596 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-cc8644fb-720b-4dfa-a2a7-3442554dfeaf tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Flavor limits 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 19 11:11:05.346793 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-cc8644fb-720b-4dfa-a2a7-3442554dfeaf tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Image limits 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 19 11:11:05.347511 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-cc8644fb-720b-4dfa-a2a7-3442554dfeaf tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Flavor pref 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 19 11:11:05.348192 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-cc8644fb-720b-4dfa-a2a7-3442554dfeaf tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Image pref 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 19 11:11:05.348869 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-cc8644fb-720b-4dfa-a2a7-3442554dfeaf tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 19 11:11:05.357966 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_api:SELECT=95,nova_api:UPDATE=1,nova_api:INSERT=1,nova_api:DELETE=1 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:11:05.366001 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cc8644fb-720b-4dfa-a2a7-3442554dfeaf tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:05.366899 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cc8644fb-720b-4dfa-a2a7-3442554dfeaf tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:05.367930 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cc8644fb-720b-4dfa-a2a7-3442554dfeaf tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:05.368572 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cc8644fb-720b-4dfa-a2a7-3442554dfeaf tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:05.392294 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.quota [None req-cc8644fb-720b-4dfa-a2a7-3442554dfeaf tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Getting quotas for project a7109adf88ac4391aee35c150e96e0b1. Resources: {'cores', 'instances', 'ram'} {{(pid=117848) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:11:05.398030 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.quota [None req-cc8644fb-720b-4dfa-a2a7-3442554dfeaf tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Getting quotas for user a217e983effa43f3a4f84221984189c5 and project a7109adf88ac4391aee35c150e96e0b1. Resources: {'cores', 'instances', 'ram'} {{(pid=117848) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:11:05.464650 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-cc8644fb-720b-4dfa-a2a7-3442554dfeaf tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Going to run 1 instances... {{(pid=117848) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 19 11:11:05.467864 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-cc8644fb-720b-4dfa-a2a7-3442554dfeaf tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='2125ea25-7fd0-42f0-ace7-b5ad9ed45ebd',volume_size=None,volume_type=None)] {{(pid=117848) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 19 11:11:05.625187 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-cc8644fb-720b-4dfa-a2a7-3442554dfeaf tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.373048 Jul 19 11:11:05.626222 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 508/996] 10.222.0.33 () {60 vars in 1146 bytes} [Tue Jul 19 11:11:04 2022] POST /compute/v2.1/servers => generated 406 bytes in 1375 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 19 11:11:05.636102 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-d2638487-e5d6-420a-b296-7d5499cf33e8 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:05.637612 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-d2638487-e5d6-420a-b296-7d5499cf33e8 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:05.706960 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-d2638487-e5d6-420a-b296-7d5499cf33e8 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:05.714458 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-d2638487-e5d6-420a-b296-7d5499cf33e8 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1095 microversion: 2.1 time: 0.082010 Jul 19 11:11:05.715425 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 489/997] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:05 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1095 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:06.353646 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-e935a764-5981-4d77-ba10-70b302c59c14 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:06.354935 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-e935a764-5981-4d77-ba10-70b302c59c14 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:06.362567 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e935a764-5981-4d77-ba10-70b302c59c14 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:06.363018 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e935a764-5981-4d77-ba10-70b302c59c14 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:06.363786 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e935a764-5981-4d77-ba10-70b302c59c14 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:06.364221 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e935a764-5981-4d77-ba10-70b302c59c14 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:06.517931 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-e935a764-5981-4d77-ba10-70b302c59c14 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:06.529797 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-e935a764-5981-4d77-ba10-70b302c59c14 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:06.531636 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-e935a764-5981-4d77-ba10-70b302c59c14 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:06.532826 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-e935a764-5981-4d77-ba10-70b302c59c14 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1739 microversion: 2.20 time: 0.183861 Jul 19 11:11:06.533621 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 509/998] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:06 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1739 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:06.732909 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-32c7e02d-867a-4db5-a09c-f4784debc234 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:06.734097 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-32c7e02d-867a-4db5-a09c-f4784debc234 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:06.747371 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-32c7e02d-867a-4db5-a09c-f4784debc234 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:06.751695 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-32c7e02d-867a-4db5-a09c-f4784debc234 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:06.753422 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-32c7e02d-867a-4db5-a09c-f4784debc234 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:06.757168 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-32c7e02d-867a-4db5-a09c-f4784debc234 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:06.766146 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-11d98578-bed4-4e3c-89c2-0ac491e55cf9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:06.768868 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-11d98578-bed4-4e3c-89c2-0ac491e55cf9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:06.783875 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-11d98578-bed4-4e3c-89c2-0ac491e55cf9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:06.784574 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-11d98578-bed4-4e3c-89c2-0ac491e55cf9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:06.785546 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-11d98578-bed4-4e3c-89c2-0ac491e55cf9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:06.789434 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-11d98578-bed4-4e3c-89c2-0ac491e55cf9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:06.864583 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-11d98578-bed4-4e3c-89c2-0ac491e55cf9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/ips" status: 200 len: 150 microversion: 2.1 time: 0.106263 Jul 19 11:11:06.872209 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 510/999] 10.222.0.33 () {58 vars in 1247 bytes} [Tue Jul 19 11:11:06 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/ips => generated 150 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:11:06.901558 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-32c7e02d-867a-4db5-a09c-f4784debc234 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:06.915748 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-32c7e02d-867a-4db5-a09c-f4784debc234 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1201 microversion: 2.1 time: 0.185444 Jul 19 11:11:06.926519 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 490/1000] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:06 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1201 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:07.566416 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-08440ced-c26e-431c-87bb-149f83d9cdb9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:07.567016 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-08440ced-c26e-431c-87bb-149f83d9cdb9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:07.574252 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-08440ced-c26e-431c-87bb-149f83d9cdb9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:07.574874 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-08440ced-c26e-431c-87bb-149f83d9cdb9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:07.576459 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-08440ced-c26e-431c-87bb-149f83d9cdb9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:07.576928 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-08440ced-c26e-431c-87bb-149f83d9cdb9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:07.739693 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-08440ced-c26e-431c-87bb-149f83d9cdb9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:07.751940 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-08440ced-c26e-431c-87bb-149f83d9cdb9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:07.755475 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-08440ced-c26e-431c-87bb-149f83d9cdb9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:07.765449 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-08440ced-c26e-431c-87bb-149f83d9cdb9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1739 microversion: 2.20 time: 0.200828 Jul 19 11:11:07.771316 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 511/1001] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:07 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1739 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:07.933038 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-8dbdb0d2-e32b-4c47-8ae1-83720b77fb95 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:07.933646 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-8dbdb0d2-e32b-4c47-8ae1-83720b77fb95 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:07.943599 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8dbdb0d2-e32b-4c47-8ae1-83720b77fb95 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:07.944055 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8dbdb0d2-e32b-4c47-8ae1-83720b77fb95 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:07.944877 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8dbdb0d2-e32b-4c47-8ae1-83720b77fb95 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:07.945497 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8dbdb0d2-e32b-4c47-8ae1-83720b77fb95 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:08.263845 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-8dbdb0d2-e32b-4c47-8ae1-83720b77fb95 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:08.276301 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-8dbdb0d2-e32b-4c47-8ae1-83720b77fb95 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1279 microversion: 2.1 time: 0.347073 Jul 19 11:11:08.286420 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 491/1002] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:07 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1279 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:08.470323 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-797838fb-4087-422e-8bfc-99d21fd68680 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "b46b33e4-95f0-4b5e-a565-0ab404ce6b3d"}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:11:08.471523 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-797838fb-4087-422e-8bfc-99d21fd68680 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:08.479005 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-797838fb-4087-422e-8bfc-99d21fd68680 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:08.479465 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-797838fb-4087-422e-8bfc-99d21fd68680 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:08.480274 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-797838fb-4087-422e-8bfc-99d21fd68680 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:08.480753 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-797838fb-4087-422e-8bfc-99d21fd68680 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:08.790286 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-0a2bd941-9e84-4b88-a881-52b04800bdc0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:08.791630 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-0a2bd941-9e84-4b88-a881-52b04800bdc0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:08.807755 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0a2bd941-9e84-4b88-a881-52b04800bdc0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:08.808219 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0a2bd941-9e84-4b88-a881-52b04800bdc0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:08.812387 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0a2bd941-9e84-4b88-a881-52b04800bdc0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:08.813802 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0a2bd941-9e84-4b88-a881-52b04800bdc0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:08.896096 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-797838fb-4087-422e-8bfc-99d21fd68680 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "POST /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.430496 Jul 19 11:11:08.897112 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 512/1003] 10.222.0.33 () {60 vars in 1322 bytes} [Tue Jul 19 11:11:08 2022] POST /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments => generated 194 bytes in 432 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:11:09.050213 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-0a2bd941-9e84-4b88-a881-52b04800bdc0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:09.058776 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-0a2bd941-9e84-4b88-a881-52b04800bdc0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:09.060383 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-0a2bd941-9e84-4b88-a881-52b04800bdc0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:09.061657 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-0a2bd941-9e84-4b88-a881-52b04800bdc0 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1739 microversion: 2.20 time: 0.276008 Jul 19 11:11:09.062239 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 492/1004] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:08 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1739 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:09.138939 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-a57f5136-79a1-4f58-9043-ad78164753ed req-0dbb8152-9685-4450-8662-9adea3132f2c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2", "tag": "29b83b99-dced-4119-b3f1-cb6ec3d0c385"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:11:09.151151 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-a57f5136-79a1-4f58-9043-ad78164753ed req-0dbb8152-9685-4450-8662-9adea3132f2c service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:09.151570 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-a57f5136-79a1-4f58-9043-ad78164753ed req-0dbb8152-9685-4450-8662-9adea3132f2c service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:09.168425 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-a57f5136-79a1-4f58-9043-ad78164753ed req-0dbb8152-9685-4450-8662-9adea3132f2c service nova] Creating event network-changed:29b83b99-dced-4119-b3f1-cb6ec3d0c385 for instance 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 on hv-cinder-maste Jul 19 11:11:09.179999 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-a57f5136-79a1-4f58-9043-ad78164753ed req-0dbb8152-9685-4450-8662-9adea3132f2c service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045990 Jul 19 11:11:09.185017 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 513/1005] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:11:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:11:09.293603 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-893ce2fd-ff46-4138-be0a-618e9b66dcdc tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:09.294255 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-893ce2fd-ff46-4138-be0a-618e9b66dcdc tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:09.300574 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-893ce2fd-ff46-4138-be0a-618e9b66dcdc tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:09.301067 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-893ce2fd-ff46-4138-be0a-618e9b66dcdc tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:09.301851 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-893ce2fd-ff46-4138-be0a-618e9b66dcdc tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:09.302288 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-893ce2fd-ff46-4138-be0a-618e9b66dcdc tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:09.506232 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-893ce2fd-ff46-4138-be0a-618e9b66dcdc tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:09.514553 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-893ce2fd-ff46-4138-be0a-618e9b66dcdc tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.224822 Jul 19 11:11:09.523881 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 493/1006] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:09 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:10.081123 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-e7e76137-3343-4a46-a4da-c92cd33b4531 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:10.081822 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-e7e76137-3343-4a46-a4da-c92cd33b4531 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:10.087944 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e7e76137-3343-4a46-a4da-c92cd33b4531 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:10.088392 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e7e76137-3343-4a46-a4da-c92cd33b4531 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:10.089207 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e7e76137-3343-4a46-a4da-c92cd33b4531 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:10.089640 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e7e76137-3343-4a46-a4da-c92cd33b4531 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:10.228677 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-e7e76137-3343-4a46-a4da-c92cd33b4531 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:10.237136 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-e7e76137-3343-4a46-a4da-c92cd33b4531 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:10.238644 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-e7e76137-3343-4a46-a4da-c92cd33b4531 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:10.239792 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-e7e76137-3343-4a46-a4da-c92cd33b4531 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1721 microversion: 2.20 time: 0.163842 Jul 19 11:11:10.240345 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 514/1007] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:10 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1721 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:10.533484 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-02573734-18ef-48e7-adb8-fcc6805d40a1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:10.534357 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-02573734-18ef-48e7-adb8-fcc6805d40a1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:10.541842 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-02573734-18ef-48e7-adb8-fcc6805d40a1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:10.542292 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-02573734-18ef-48e7-adb8-fcc6805d40a1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:10.543061 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-02573734-18ef-48e7-adb8-fcc6805d40a1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:10.543681 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-02573734-18ef-48e7-adb8-fcc6805d40a1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:10.694696 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-02573734-18ef-48e7-adb8-fcc6805d40a1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:10.703489 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-02573734-18ef-48e7-adb8-fcc6805d40a1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.175095 Jul 19 11:11:10.704474 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 494/1008] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:10 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:11.259318 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-bea0148a-2e38-4945-b4da-7641353cb03e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:11.259880 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-bea0148a-2e38-4945-b4da-7641353cb03e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:11.266353 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-bea0148a-2e38-4945-b4da-7641353cb03e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:11.266807 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-bea0148a-2e38-4945-b4da-7641353cb03e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:11.267605 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-bea0148a-2e38-4945-b4da-7641353cb03e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:11.268041 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-bea0148a-2e38-4945-b4da-7641353cb03e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:11.401100 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-bea0148a-2e38-4945-b4da-7641353cb03e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:11.409210 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-bea0148a-2e38-4945-b4da-7641353cb03e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:11.410727 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-bea0148a-2e38-4945-b4da-7641353cb03e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:11.411872 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-bea0148a-2e38-4945-b4da-7641353cb03e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1721 microversion: 2.20 time: 0.156557 Jul 19 11:11:11.412425 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 515/1009] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:11 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1721 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:11.725343 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-59a0bd39-07e7-4e4a-bc38-6977c0e0d4ae tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:11.726665 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-59a0bd39-07e7-4e4a-bc38-6977c0e0d4ae tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:11.741155 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-59a0bd39-07e7-4e4a-bc38-6977c0e0d4ae tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:11.741610 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-59a0bd39-07e7-4e4a-bc38-6977c0e0d4ae tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:11.742399 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-59a0bd39-07e7-4e4a-bc38-6977c0e0d4ae tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:11.742844 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-59a0bd39-07e7-4e4a-bc38-6977c0e0d4ae tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:11.915188 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-59a0bd39-07e7-4e4a-bc38-6977c0e0d4ae tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:11.933566 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-59a0bd39-07e7-4e4a-bc38-6977c0e0d4ae tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.212548 Jul 19 11:11:11.935229 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 495/1010] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:11 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:12.430279 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-8a7d8e9d-871e-4850-bfff-60fa0080ab5b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:12.431628 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-8a7d8e9d-871e-4850-bfff-60fa0080ab5b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:12.439386 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8a7d8e9d-871e-4850-bfff-60fa0080ab5b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:12.439946 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8a7d8e9d-871e-4850-bfff-60fa0080ab5b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:12.440988 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8a7d8e9d-871e-4850-bfff-60fa0080ab5b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:12.441629 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8a7d8e9d-871e-4850-bfff-60fa0080ab5b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:12.445848 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:11:12.581650 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-8a7d8e9d-871e-4850-bfff-60fa0080ab5b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:12.589660 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-8a7d8e9d-871e-4850-bfff-60fa0080ab5b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:12.591258 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-8a7d8e9d-871e-4850-bfff-60fa0080ab5b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:12.592402 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-8a7d8e9d-871e-4850-bfff-60fa0080ab5b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1664 microversion: 2.20 time: 0.166285 Jul 19 11:11:12.592995 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 516/1011] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:12 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1664 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:12.610382 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-d6763d14-3642-4ec9-b3b0-00c87d57c463 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "e84cf673-e4dc-469a-ab49-9658f48a509f"}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:11:12.611486 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-d6763d14-3642-4ec9-b3b0-00c87d57c463 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:12.613688 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_api:SELECT=93,nova_api:DELETE=1,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:11:12.619186 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d6763d14-3642-4ec9-b3b0-00c87d57c463 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:12.619857 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d6763d14-3642-4ec9-b3b0-00c87d57c463 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:12.620634 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d6763d14-3642-4ec9-b3b0-00c87d57c463 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:12.621100 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d6763d14-3642-4ec9-b3b0-00c87d57c463 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:12.710781 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-a8185501-88ad-49c2-932d-17915d19f74d req-44d06df8-a3b7-47c6-8ffe-de657d4fabfc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "36abf2ae-3b4c-4095-b9ba-2bd738e266d9", "name": "network-vif-unplugged", "status": "completed", "tag": "049ef62d-5f12-4134-9010-2bc20edecd41"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:11:12.724111 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-a8185501-88ad-49c2-932d-17915d19f74d req-44d06df8-a3b7-47c6-8ffe-de657d4fabfc service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:12.724722 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-a8185501-88ad-49c2-932d-17915d19f74d req-44d06df8-a3b7-47c6-8ffe-de657d4fabfc service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:12.744172 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.compute.server_external_events [req-a8185501-88ad-49c2-932d-17915d19f74d req-44d06df8-a3b7-47c6-8ffe-de657d4fabfc service nova] Unable to find a host for instance 36abf2ae-3b4c-4095-b9ba-2bd738e266d9. Dropping event network-vif-unplugged {{(pid=117848) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:143}} Jul 19 11:11:12.745318 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-a8185501-88ad-49c2-932d-17915d19f74d req-44d06df8-a3b7-47c6-8ffe-de657d4fabfc service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.038427 Jul 19 11:11:12.746314 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 517/1012] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:11:12 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 40 msecs (HTTP/1.1 207) 9 headers in 367 bytes (1 switches on core 0) Jul 19 11:11:12.931845 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-d6763d14-3642-4ec9-b3b0-00c87d57c463 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "POST /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.325450 Jul 19 11:11:12.932587 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 496/1013] 10.222.0.33 () {62 vars in 1363 bytes} [Tue Jul 19 11:11:12 2022] POST /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9/os-volume_attachments => generated 188 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 19 11:11:12.951039 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-9976ca82-cd25-4fd5-83fb-038c2972b0f6 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:12.951648 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-9976ca82-cd25-4fd5-83fb-038c2972b0f6 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:12.957721 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9976ca82-cd25-4fd5-83fb-038c2972b0f6 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:12.958174 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9976ca82-cd25-4fd5-83fb-038c2972b0f6 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:12.958973 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9976ca82-cd25-4fd5-83fb-038c2972b0f6 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:12.959411 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9976ca82-cd25-4fd5-83fb-038c2972b0f6 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:13.065332 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-4b738bda-690b-475c-9db9-8dcc0b1b4c98 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:13.065769 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-4b738bda-690b-475c-9db9-8dcc0b1b4c98 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:13.073318 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4b738bda-690b-475c-9db9-8dcc0b1b4c98 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:13.073846 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4b738bda-690b-475c-9db9-8dcc0b1b4c98 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:13.074997 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4b738bda-690b-475c-9db9-8dcc0b1b4c98 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:13.076055 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4b738bda-690b-475c-9db9-8dcc0b1b4c98 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:13.137200 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-9976ca82-cd25-4fd5-83fb-038c2972b0f6 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:13.147134 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-9976ca82-cd25-4fd5-83fb-038c2972b0f6 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.200618 Jul 19 11:11:13.148809 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 518/1014] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:12 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:13.355646 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-4b738bda-690b-475c-9db9-8dcc0b1b4c98 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "DELETE /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9/os-volume_attachments/e84cf673-e4dc-469a-ab49-9658f48a509f" status: 202 len: 0 microversion: 2.20 time: 0.294305 Jul 19 11:11:13.356358 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 497/1015] 10.222.0.33 () {60 vars in 1456 bytes} [Tue Jul 19 11:11:13 2022] DELETE /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9/os-volume_attachments/e84cf673-e4dc-469a-ab49-9658f48a509f => generated 0 bytes in 296 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Jul 19 11:11:13.411023 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-33807664-e0a9-4d10-a82c-456f014ec9d8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:11:13.411591 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-33807664-e0a9-4d10-a82c-456f014ec9d8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:13.417667 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-33807664-e0a9-4d10-a82c-456f014ec9d8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:13.418099 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-33807664-e0a9-4d10-a82c-456f014ec9d8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:13.418960 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-33807664-e0a9-4d10-a82c-456f014ec9d8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:13.419419 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-33807664-e0a9-4d10-a82c-456f014ec9d8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:13.506853 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-33807664-e0a9-4d10-a82c-456f014ec9d8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "POST /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9/action" status: 202 len: 0 microversion: 2.20 time: 0.099611 Jul 19 11:11:13.510363 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 519/1016] 10.222.0.33 () {62 vars in 1318 bytes} [Tue Jul 19 11:11:13 2022] POST /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9/action => generated 0 bytes in 101 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 19 11:11:13.516263 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-7f9a3248-1b1f-4f18-9246-099265a24344 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:13.516980 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-7f9a3248-1b1f-4f18-9246-099265a24344 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:13.525079 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7f9a3248-1b1f-4f18-9246-099265a24344 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:13.526414 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7f9a3248-1b1f-4f18-9246-099265a24344 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:13.527718 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7f9a3248-1b1f-4f18-9246-099265a24344 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:13.528720 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7f9a3248-1b1f-4f18-9246-099265a24344 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:13.557325 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=87,nova_cell1:INSERT=5,nova_cell1:UPDATE=8 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:11:13.688611 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-7f9a3248-1b1f-4f18-9246-099265a24344 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:13.696663 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-7f9a3248-1b1f-4f18-9246-099265a24344 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:13.698220 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-7f9a3248-1b1f-4f18-9246-099265a24344 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:13.699370 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-7f9a3248-1b1f-4f18-9246-099265a24344 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1672 microversion: 2.20 time: 0.187665 Jul 19 11:11:13.700215 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 498/1017] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:13 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1672 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:14.166048 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-f08871a8-2794-4ae4-8e7b-411f437426c5 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:14.166778 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-f08871a8-2794-4ae4-8e7b-411f437426c5 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:14.173462 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f08871a8-2794-4ae4-8e7b-411f437426c5 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:14.173910 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f08871a8-2794-4ae4-8e7b-411f437426c5 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:14.174671 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f08871a8-2794-4ae4-8e7b-411f437426c5 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:14.175246 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f08871a8-2794-4ae4-8e7b-411f437426c5 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:14.328058 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-f08871a8-2794-4ae4-8e7b-411f437426c5 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:14.335462 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-f08871a8-2794-4ae4-8e7b-411f437426c5 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.173325 Jul 19 11:11:14.336027 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 520/1018] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:14 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:14.718426 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-9beed382-547a-44ec-afed-f2d1a65042a6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:14.719160 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-9beed382-547a-44ec-afed-f2d1a65042a6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:14.726149 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9beed382-547a-44ec-afed-f2d1a65042a6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:14.726616 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9beed382-547a-44ec-afed-f2d1a65042a6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:14.727381 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9beed382-547a-44ec-afed-f2d1a65042a6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:14.727817 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9beed382-547a-44ec-afed-f2d1a65042a6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:14.857695 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-9beed382-547a-44ec-afed-f2d1a65042a6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:14.865558 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-9beed382-547a-44ec-afed-f2d1a65042a6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:14.867108 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-9beed382-547a-44ec-afed-f2d1a65042a6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:14.868251 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-9beed382-547a-44ec-afed-f2d1a65042a6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.154281 Jul 19 11:11:14.869007 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 499/1019] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:14 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:15.354105 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-926cd738-544e-44cf-8e01-4fb862faaf4f tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:15.354708 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-926cd738-544e-44cf-8e01-4fb862faaf4f tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:15.363519 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-926cd738-544e-44cf-8e01-4fb862faaf4f tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:15.363965 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-926cd738-544e-44cf-8e01-4fb862faaf4f tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:15.364750 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-926cd738-544e-44cf-8e01-4fb862faaf4f tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:15.365186 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-926cd738-544e-44cf-8e01-4fb862faaf4f tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:15.381194 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell0:SELECT=2 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:11:15.407995 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell0:SELECT=6 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:11:15.501537 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-926cd738-544e-44cf-8e01-4fb862faaf4f tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:15.509981 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-926cd738-544e-44cf-8e01-4fb862faaf4f tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.159696 Jul 19 11:11:15.510547 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 521/1020] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:15 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:15.896017 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-a8b94916-6a4a-4b11-8e10-b1b4e0e6b917 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:15.905041 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-a8b94916-6a4a-4b11-8e10-b1b4e0e6b917 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:15.915545 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell0:SELECT=2 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:11:15.935551 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a8b94916-6a4a-4b11-8e10-b1b4e0e6b917 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:15.936368 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a8b94916-6a4a-4b11-8e10-b1b4e0e6b917 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:15.937542 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a8b94916-6a4a-4b11-8e10-b1b4e0e6b917 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:15.938243 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a8b94916-6a4a-4b11-8e10-b1b4e0e6b917 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:16.127619 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-a8b94916-6a4a-4b11-8e10-b1b4e0e6b917 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:16.138442 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-a8b94916-6a4a-4b11-8e10-b1b4e0e6b917 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:16.140380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-a8b94916-6a4a-4b11-8e10-b1b4e0e6b917 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:16.141771 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-a8b94916-6a4a-4b11-8e10-b1b4e0e6b917 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.257646 Jul 19 11:11:16.142356 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 500/1021] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:15 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:16.526994 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-51297d52-8ad8-4e08-bc26-d63ea7cd91ba tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:16.528338 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-51297d52-8ad8-4e08-bc26-d63ea7cd91ba tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:16.537009 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-51297d52-8ad8-4e08-bc26-d63ea7cd91ba tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:16.537552 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-51297d52-8ad8-4e08-bc26-d63ea7cd91ba tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:16.538348 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-51297d52-8ad8-4e08-bc26-d63ea7cd91ba tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:16.538971 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-51297d52-8ad8-4e08-bc26-d63ea7cd91ba tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:16.695738 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-51297d52-8ad8-4e08-bc26-d63ea7cd91ba tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:16.705826 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-51297d52-8ad8-4e08-bc26-d63ea7cd91ba tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.182979 Jul 19 11:11:16.708564 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 522/1022] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:16 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:17.160364 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-da6d5244-d030-4cb3-97ad-15d46323fd89 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:17.161736 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-da6d5244-d030-4cb3-97ad-15d46323fd89 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:17.171220 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-da6d5244-d030-4cb3-97ad-15d46323fd89 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:17.171769 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-da6d5244-d030-4cb3-97ad-15d46323fd89 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:17.172781 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-da6d5244-d030-4cb3-97ad-15d46323fd89 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:17.174138 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-da6d5244-d030-4cb3-97ad-15d46323fd89 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:17.349314 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-da6d5244-d030-4cb3-97ad-15d46323fd89 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:17.357854 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-da6d5244-d030-4cb3-97ad-15d46323fd89 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:17.359377 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-da6d5244-d030-4cb3-97ad-15d46323fd89 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:17.360587 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-da6d5244-d030-4cb3-97ad-15d46323fd89 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.204208 Jul 19 11:11:17.361413 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 501/1023] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:17 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:17.726165 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-b6daa869-430e-44ea-bd0a-c78bad0a5d3c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:17.727159 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-b6daa869-430e-44ea-bd0a-c78bad0a5d3c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:17.734447 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b6daa869-430e-44ea-bd0a-c78bad0a5d3c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:17.735040 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b6daa869-430e-44ea-bd0a-c78bad0a5d3c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:17.738849 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b6daa869-430e-44ea-bd0a-c78bad0a5d3c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:17.739424 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b6daa869-430e-44ea-bd0a-c78bad0a5d3c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:17.898702 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-b6daa869-430e-44ea-bd0a-c78bad0a5d3c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:17.923046 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-b6daa869-430e-44ea-bd0a-c78bad0a5d3c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.201808 Jul 19 11:11:17.923639 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 523/1024] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:17 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:18.380078 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-b4da140d-d636-43b1-b12d-20f40e379ea5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:18.380856 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-b4da140d-d636-43b1-b12d-20f40e379ea5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:18.387464 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b4da140d-d636-43b1-b12d-20f40e379ea5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:18.387916 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b4da140d-d636-43b1-b12d-20f40e379ea5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:18.388690 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b4da140d-d636-43b1-b12d-20f40e379ea5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:18.391161 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b4da140d-d636-43b1-b12d-20f40e379ea5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:18.538073 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b4da140d-d636-43b1-b12d-20f40e379ea5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:18.548055 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b4da140d-d636-43b1-b12d-20f40e379ea5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:18.549633 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b4da140d-d636-43b1-b12d-20f40e379ea5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:18.550798 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-b4da140d-d636-43b1-b12d-20f40e379ea5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.174635 Jul 19 11:11:18.551673 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 502/1025] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:18 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:18.961192 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-c1a64c84-b04e-4503-9a38-4168069c43bd tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:18.962877 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-c1a64c84-b04e-4503-9a38-4168069c43bd tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:18.972805 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c1a64c84-b04e-4503-9a38-4168069c43bd tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:18.973842 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c1a64c84-b04e-4503-9a38-4168069c43bd tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:18.974617 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c1a64c84-b04e-4503-9a38-4168069c43bd tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:18.975269 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c1a64c84-b04e-4503-9a38-4168069c43bd tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:19.125884 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-c1a64c84-b04e-4503-9a38-4168069c43bd tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:19.133355 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-c1a64c84-b04e-4503-9a38-4168069c43bd tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.196840 Jul 19 11:11:19.133911 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 524/1026] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:18 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:19.568836 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-59b3576d-1890-4a6c-8a7e-903b3a9bb922 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:19.569444 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-59b3576d-1890-4a6c-8a7e-903b3a9bb922 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:19.582271 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-59b3576d-1890-4a6c-8a7e-903b3a9bb922 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:19.583348 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-59b3576d-1890-4a6c-8a7e-903b3a9bb922 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:19.584768 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-59b3576d-1890-4a6c-8a7e-903b3a9bb922 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:19.585576 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-59b3576d-1890-4a6c-8a7e-903b3a9bb922 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:19.746403 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-59b3576d-1890-4a6c-8a7e-903b3a9bb922 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:19.756149 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-59b3576d-1890-4a6c-8a7e-903b3a9bb922 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:19.758003 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-59b3576d-1890-4a6c-8a7e-903b3a9bb922 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:19.761025 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-59b3576d-1890-4a6c-8a7e-903b3a9bb922 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.195468 Jul 19 11:11:19.763395 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 503/1027] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:19 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:20.150884 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-897c7231-2bbd-4ba5-8811-2789f97d32fd tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:20.152198 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-897c7231-2bbd-4ba5-8811-2789f97d32fd tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:20.160806 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-897c7231-2bbd-4ba5-8811-2789f97d32fd tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:20.163495 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-897c7231-2bbd-4ba5-8811-2789f97d32fd tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:20.165464 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-897c7231-2bbd-4ba5-8811-2789f97d32fd tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:20.166150 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-897c7231-2bbd-4ba5-8811-2789f97d32fd tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:20.308199 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-897c7231-2bbd-4ba5-8811-2789f97d32fd tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:20.315615 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-897c7231-2bbd-4ba5-8811-2789f97d32fd tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.168715 Jul 19 11:11:20.316179 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 525/1028] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:20 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:20.781513 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-4c9f19a8-2301-4477-a18c-8530ab60e584 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:20.782757 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-4c9f19a8-2301-4477-a18c-8530ab60e584 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:20.792195 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4c9f19a8-2301-4477-a18c-8530ab60e584 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:20.792582 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4c9f19a8-2301-4477-a18c-8530ab60e584 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:20.793417 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4c9f19a8-2301-4477-a18c-8530ab60e584 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:20.793857 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4c9f19a8-2301-4477-a18c-8530ab60e584 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:20.952097 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-4c9f19a8-2301-4477-a18c-8530ab60e584 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:20.960104 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-4c9f19a8-2301-4477-a18c-8530ab60e584 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:20.961636 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-4c9f19a8-2301-4477-a18c-8530ab60e584 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:20.962784 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-4c9f19a8-2301-4477-a18c-8530ab60e584 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.185367 Jul 19 11:11:20.963509 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 504/1029] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:20 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:21.333016 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-53458d2b-b1da-4307-8553-b2ad996a0a0b tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:21.334032 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-53458d2b-b1da-4307-8553-b2ad996a0a0b tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:21.341490 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-53458d2b-b1da-4307-8553-b2ad996a0a0b tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:21.342355 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-53458d2b-b1da-4307-8553-b2ad996a0a0b tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:21.343146 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-53458d2b-b1da-4307-8553-b2ad996a0a0b tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:21.343587 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-53458d2b-b1da-4307-8553-b2ad996a0a0b tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:21.494768 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-53458d2b-b1da-4307-8553-b2ad996a0a0b tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:21.504024 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-53458d2b-b1da-4307-8553-b2ad996a0a0b tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.176094 Jul 19 11:11:21.505298 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 526/1030] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:21 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:21.981078 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-0027be3f-f51a-442e-adb8-2a44e8de3ff9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:21.981685 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-0027be3f-f51a-442e-adb8-2a44e8de3ff9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:21.988187 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0027be3f-f51a-442e-adb8-2a44e8de3ff9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:21.988641 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0027be3f-f51a-442e-adb8-2a44e8de3ff9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:21.989447 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0027be3f-f51a-442e-adb8-2a44e8de3ff9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:21.989890 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0027be3f-f51a-442e-adb8-2a44e8de3ff9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:22.135602 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-0027be3f-f51a-442e-adb8-2a44e8de3ff9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:22.144070 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-0027be3f-f51a-442e-adb8-2a44e8de3ff9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:22.145624 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-0027be3f-f51a-442e-adb8-2a44e8de3ff9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:22.146911 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-0027be3f-f51a-442e-adb8-2a44e8de3ff9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.169729 Jul 19 11:11:22.147470 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 505/1031] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:21 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:22.522182 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-c4f8f386-7e72-4f7d-9356-ff404ddbd618 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:22.523496 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-c4f8f386-7e72-4f7d-9356-ff404ddbd618 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:22.530114 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c4f8f386-7e72-4f7d-9356-ff404ddbd618 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:22.530556 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c4f8f386-7e72-4f7d-9356-ff404ddbd618 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:22.531546 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c4f8f386-7e72-4f7d-9356-ff404ddbd618 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:22.531978 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c4f8f386-7e72-4f7d-9356-ff404ddbd618 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:22.710206 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-c4f8f386-7e72-4f7d-9356-ff404ddbd618 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:22.729138 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-c4f8f386-7e72-4f7d-9356-ff404ddbd618 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.210860 Jul 19 11:11:22.730104 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 527/1032] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:22 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:23.165729 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-fb960439-fb44-4e63-b6b8-eee6476231ca tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:23.166474 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-fb960439-fb44-4e63-b6b8-eee6476231ca tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:23.173789 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fb960439-fb44-4e63-b6b8-eee6476231ca tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:23.174232 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fb960439-fb44-4e63-b6b8-eee6476231ca tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:23.175003 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fb960439-fb44-4e63-b6b8-eee6476231ca tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:23.175434 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fb960439-fb44-4e63-b6b8-eee6476231ca tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:23.326239 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-fb960439-fb44-4e63-b6b8-eee6476231ca tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:23.334529 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-fb960439-fb44-4e63-b6b8-eee6476231ca tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:23.336026 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-fb960439-fb44-4e63-b6b8-eee6476231ca tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:23.337200 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-fb960439-fb44-4e63-b6b8-eee6476231ca tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.175524 Jul 19 11:11:23.337752 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 506/1033] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:23 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:23.749434 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-f244f0ee-2b3e-46f8-8fea-53da90e79970 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:23.750472 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-f244f0ee-2b3e-46f8-8fea-53da90e79970 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:23.757277 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f244f0ee-2b3e-46f8-8fea-53da90e79970 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:23.757818 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f244f0ee-2b3e-46f8-8fea-53da90e79970 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:23.758655 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f244f0ee-2b3e-46f8-8fea-53da90e79970 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:23.759179 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f244f0ee-2b3e-46f8-8fea-53da90e79970 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:23.897024 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-f244f0ee-2b3e-46f8-8fea-53da90e79970 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:23.905349 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-f244f0ee-2b3e-46f8-8fea-53da90e79970 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.160296 Jul 19 11:11:23.905924 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 528/1034] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:23 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:24.355618 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-972af0f8-0c16-42e0-ba33-bd6b8c7ce90c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:24.356312 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-972af0f8-0c16-42e0-ba33-bd6b8c7ce90c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:24.363604 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-972af0f8-0c16-42e0-ba33-bd6b8c7ce90c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:24.364077 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-972af0f8-0c16-42e0-ba33-bd6b8c7ce90c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:24.364885 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-972af0f8-0c16-42e0-ba33-bd6b8c7ce90c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:24.365323 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-972af0f8-0c16-42e0-ba33-bd6b8c7ce90c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:24.498508 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-972af0f8-0c16-42e0-ba33-bd6b8c7ce90c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:24.506662 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-972af0f8-0c16-42e0-ba33-bd6b8c7ce90c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:24.508197 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-972af0f8-0c16-42e0-ba33-bd6b8c7ce90c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:24.509482 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-972af0f8-0c16-42e0-ba33-bd6b8c7ce90c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.158056 Jul 19 11:11:24.510041 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 507/1035] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:24 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:24.922630 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-dc629956-d6d3-4e5b-95ca-5c4ef787d190 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:24.923271 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-dc629956-d6d3-4e5b-95ca-5c4ef787d190 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:24.929727 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dc629956-d6d3-4e5b-95ca-5c4ef787d190 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:24.930178 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dc629956-d6d3-4e5b-95ca-5c4ef787d190 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:24.930961 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dc629956-d6d3-4e5b-95ca-5c4ef787d190 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:24.931393 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dc629956-d6d3-4e5b-95ca-5c4ef787d190 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:25.064326 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-dc629956-d6d3-4e5b-95ca-5c4ef787d190 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:25.072303 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-dc629956-d6d3-4e5b-95ca-5c4ef787d190 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.153707 Jul 19 11:11:25.073253 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 529/1036] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:24 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:25.527530 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-912dfe52-84ff-43e3-9e57-bbf950a3103a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:25.528135 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-912dfe52-84ff-43e3-9e57-bbf950a3103a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:25.534693 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-912dfe52-84ff-43e3-9e57-bbf950a3103a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:25.535144 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-912dfe52-84ff-43e3-9e57-bbf950a3103a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:25.535902 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-912dfe52-84ff-43e3-9e57-bbf950a3103a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:25.536343 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-912dfe52-84ff-43e3-9e57-bbf950a3103a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:25.674175 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-912dfe52-84ff-43e3-9e57-bbf950a3103a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:25.682100 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-912dfe52-84ff-43e3-9e57-bbf950a3103a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:25.683604 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-912dfe52-84ff-43e3-9e57-bbf950a3103a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:25.684775 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-912dfe52-84ff-43e3-9e57-bbf950a3103a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.161156 Jul 19 11:11:25.685988 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 508/1037] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:25 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:26.089188 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-835ad229-f891-4ee9-a24f-f2663f41706c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:26.089726 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-835ad229-f891-4ee9-a24f-f2663f41706c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:26.096665 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-835ad229-f891-4ee9-a24f-f2663f41706c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:26.097140 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-835ad229-f891-4ee9-a24f-f2663f41706c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:26.097886 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-835ad229-f891-4ee9-a24f-f2663f41706c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:26.098320 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-835ad229-f891-4ee9-a24f-f2663f41706c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:26.225408 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-835ad229-f891-4ee9-a24f-f2663f41706c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:26.234189 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-835ad229-f891-4ee9-a24f-f2663f41706c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.148377 Jul 19 11:11:26.234780 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 530/1038] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:26 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:26.704464 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-011247db-de95-4173-a66e-49981ea1c491 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:26.705112 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-011247db-de95-4173-a66e-49981ea1c491 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:26.711644 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-011247db-de95-4173-a66e-49981ea1c491 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:26.712093 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-011247db-de95-4173-a66e-49981ea1c491 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:26.712888 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-011247db-de95-4173-a66e-49981ea1c491 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:26.713328 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-011247db-de95-4173-a66e-49981ea1c491 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:26.844112 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-011247db-de95-4173-a66e-49981ea1c491 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:26.852036 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-011247db-de95-4173-a66e-49981ea1c491 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:26.853566 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-011247db-de95-4173-a66e-49981ea1c491 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:26.854695 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-011247db-de95-4173-a66e-49981ea1c491 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.154841 Jul 19 11:11:26.855282 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 509/1039] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:26 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:27.251781 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-463afe91-dbfe-42c4-9a04-4748793c204a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:27.252324 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-463afe91-dbfe-42c4-9a04-4748793c204a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:27.258844 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-463afe91-dbfe-42c4-9a04-4748793c204a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:27.259295 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-463afe91-dbfe-42c4-9a04-4748793c204a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:27.260053 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-463afe91-dbfe-42c4-9a04-4748793c204a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:27.260488 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-463afe91-dbfe-42c4-9a04-4748793c204a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:27.390841 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-463afe91-dbfe-42c4-9a04-4748793c204a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:27.398215 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-463afe91-dbfe-42c4-9a04-4748793c204a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.151244 Jul 19 11:11:27.398779 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 531/1040] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:27 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:27.872377 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-8252f84c-5924-4dfa-b4f2-a91eeaf1928c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:27.873097 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-8252f84c-5924-4dfa-b4f2-a91eeaf1928c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:27.880437 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8252f84c-5924-4dfa-b4f2-a91eeaf1928c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:27.881137 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8252f84c-5924-4dfa-b4f2-a91eeaf1928c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:27.881989 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8252f84c-5924-4dfa-b4f2-a91eeaf1928c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:27.882493 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8252f84c-5924-4dfa-b4f2-a91eeaf1928c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:28.084092 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-8252f84c-5924-4dfa-b4f2-a91eeaf1928c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:28.092184 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-8252f84c-5924-4dfa-b4f2-a91eeaf1928c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:28.094341 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-8252f84c-5924-4dfa-b4f2-a91eeaf1928c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:28.095565 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-8252f84c-5924-4dfa-b4f2-a91eeaf1928c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.227294 Jul 19 11:11:28.096434 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 510/1041] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:27 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:28.230537 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-8c78e79c-706b-4243-8690-eceff324a8aa tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:28.231051 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-8c78e79c-706b-4243-8690-eceff324a8aa tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:28.237696 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8c78e79c-706b-4243-8690-eceff324a8aa tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:28.238143 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8c78e79c-706b-4243-8690-eceff324a8aa tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:28.238900 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8c78e79c-706b-4243-8690-eceff324a8aa tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:28.239333 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8c78e79c-706b-4243-8690-eceff324a8aa tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:28.259123 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:11:28.321140 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.objects.instance [None req-8c78e79c-706b-4243-8690-eceff324a8aa tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lazy-loading 'services' on Instance uuid e9cecc8a-4c06-454a-91b6-0b735a222e67 {{(pid=117848) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 19 11:11:28.389523 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-8c78e79c-706b-4243-8690-eceff324a8aa tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "DELETE /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments/b46b33e4-95f0-4b5e-a565-0ab404ce6b3d" status: 202 len: 0 microversion: 2.1 time: 0.162882 Jul 19 11:11:28.390099 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 532/1042] 10.222.0.33 () {58 vars in 1415 bytes} [Tue Jul 19 11:11:28 2022] DELETE /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments/b46b33e4-95f0-4b5e-a565-0ab404ce6b3d => generated 0 bytes in 164 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 19 11:11:28.398574 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-bb09d540-d398-480e-801e-67c48abed166 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:28.399667 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-bb09d540-d398-480e-801e-67c48abed166 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:28.406335 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bb09d540-d398-480e-801e-67c48abed166 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:28.406877 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bb09d540-d398-480e-801e-67c48abed166 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:28.407732 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bb09d540-d398-480e-801e-67c48abed166 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:28.408199 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bb09d540-d398-480e-801e-67c48abed166 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:28.415882 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-40941bf4-8e21-434e-81cd-70087279e484 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:28.416571 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-40941bf4-8e21-434e-81cd-70087279e484 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:28.425367 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-40941bf4-8e21-434e-81cd-70087279e484 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:28.426491 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-40941bf4-8e21-434e-81cd-70087279e484 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:28.427296 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-40941bf4-8e21-434e-81cd-70087279e484 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:28.427719 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-40941bf4-8e21-434e-81cd-70087279e484 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:28.438984 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-bb09d540-d398-480e-801e-67c48abed166 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044509 Jul 19 11:11:28.439553 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 511/1043] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:11:28 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:11:28.560304 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-40941bf4-8e21-434e-81cd-70087279e484 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:28.567861 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-40941bf4-8e21-434e-81cd-70087279e484 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.156197 Jul 19 11:11:28.568422 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 533/1044] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:28 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:11:29.115760 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-3887edf4-b4f1-44bb-ad70-20314bfea9c3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:29.116516 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-3887edf4-b4f1-44bb-ad70-20314bfea9c3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:29.129075 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3887edf4-b4f1-44bb-ad70-20314bfea9c3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:29.129075 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3887edf4-b4f1-44bb-ad70-20314bfea9c3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:29.129075 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3887edf4-b4f1-44bb-ad70-20314bfea9c3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:29.129075 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3887edf4-b4f1-44bb-ad70-20314bfea9c3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:29.286586 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-3887edf4-b4f1-44bb-ad70-20314bfea9c3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:29.294885 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-3887edf4-b4f1-44bb-ad70-20314bfea9c3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:29.296437 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-3887edf4-b4f1-44bb-ad70-20314bfea9c3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:29.297609 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-3887edf4-b4f1-44bb-ad70-20314bfea9c3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.186230 Jul 19 11:11:29.298165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 512/1045] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:29 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:29.450385 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-8f0142f0-34b5-47a6-b2a9-cd38f96143ec tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:29.454292 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-8f0142f0-34b5-47a6-b2a9-cd38f96143ec tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:29.461780 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8f0142f0-34b5-47a6-b2a9-cd38f96143ec tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:29.462669 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8f0142f0-34b5-47a6-b2a9-cd38f96143ec tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:29.464049 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8f0142f0-34b5-47a6-b2a9-cd38f96143ec tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:29.464721 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8f0142f0-34b5-47a6-b2a9-cd38f96143ec tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:29.501405 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-8f0142f0-34b5-47a6-b2a9-cd38f96143ec tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055545 Jul 19 11:11:29.501983 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 534/1046] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:11:29 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:11:29.584247 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-5617d3b7-a2a9-44c9-93e5-d6518cefe794 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:29.584892 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-5617d3b7-a2a9-44c9-93e5-d6518cefe794 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:29.590888 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5617d3b7-a2a9-44c9-93e5-d6518cefe794 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:29.591284 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5617d3b7-a2a9-44c9-93e5-d6518cefe794 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:29.592041 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5617d3b7-a2a9-44c9-93e5-d6518cefe794 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:29.592476 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5617d3b7-a2a9-44c9-93e5-d6518cefe794 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:29.721364 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-5617d3b7-a2a9-44c9-93e5-d6518cefe794 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:29.728815 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-5617d3b7-a2a9-44c9-93e5-d6518cefe794 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.148422 Jul 19 11:11:29.729375 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 513/1047] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:29 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:30.316036 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-a9ecf945-4aeb-4d07-a4ce-2c485b301bbb tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:30.316575 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-a9ecf945-4aeb-4d07-a4ce-2c485b301bbb tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:30.322754 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a9ecf945-4aeb-4d07-a4ce-2c485b301bbb tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:30.323198 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a9ecf945-4aeb-4d07-a4ce-2c485b301bbb tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:30.323986 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a9ecf945-4aeb-4d07-a4ce-2c485b301bbb tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:30.324419 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a9ecf945-4aeb-4d07-a4ce-2c485b301bbb tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:30.452404 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-a9ecf945-4aeb-4d07-a4ce-2c485b301bbb tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:30.460359 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-a9ecf945-4aeb-4d07-a4ce-2c485b301bbb tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:30.461907 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-a9ecf945-4aeb-4d07-a4ce-2c485b301bbb tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:30.463046 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-a9ecf945-4aeb-4d07-a4ce-2c485b301bbb tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.151672 Jul 19 11:11:30.463596 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 535/1048] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:30 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:30.512752 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-776ef043-0160-4417-9856-f0fde3b84e0e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:30.513682 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-776ef043-0160-4417-9856-f0fde3b84e0e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:30.520029 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-776ef043-0160-4417-9856-f0fde3b84e0e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:30.520481 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-776ef043-0160-4417-9856-f0fde3b84e0e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:30.521285 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-776ef043-0160-4417-9856-f0fde3b84e0e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:30.521731 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-776ef043-0160-4417-9856-f0fde3b84e0e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:30.551733 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-776ef043-0160-4417-9856-f0fde3b84e0e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042938 Jul 19 11:11:30.552293 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 514/1049] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:11:30 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:11:30.745422 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-89288d11-7f8e-44dd-8138-d38eca7101bb tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:30.746207 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-89288d11-7f8e-44dd-8138-d38eca7101bb tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:30.754870 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-89288d11-7f8e-44dd-8138-d38eca7101bb tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:30.755310 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-89288d11-7f8e-44dd-8138-d38eca7101bb tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:30.756265 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-89288d11-7f8e-44dd-8138-d38eca7101bb tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:30.756892 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-89288d11-7f8e-44dd-8138-d38eca7101bb tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:30.893503 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-89288d11-7f8e-44dd-8138-d38eca7101bb tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:30.901553 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-89288d11-7f8e-44dd-8138-d38eca7101bb tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.159995 Jul 19 11:11:30.902120 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 536/1050] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:30 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:31.057865 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-2361c478-9150-4d58-bd1b-aa7dd60c18b7 req-2306a818-f426-4830-88ec-e8d46fb9c270 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2", "name": "network-vif-plugged", "status": "completed", "tag": "29b83b99-dced-4119-b3f1-cb6ec3d0c385"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:11:31.067368 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-2361c478-9150-4d58-bd1b-aa7dd60c18b7 req-2306a818-f426-4830-88ec-e8d46fb9c270 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:31.067763 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-2361c478-9150-4d58-bd1b-aa7dd60c18b7 req-2306a818-f426-4830-88ec-e8d46fb9c270 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:31.072626 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=100 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:11:31.082897 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-2361c478-9150-4d58-bd1b-aa7dd60c18b7 req-2306a818-f426-4830-88ec-e8d46fb9c270 service nova] Creating event network-vif-plugged:29b83b99-dced-4119-b3f1-cb6ec3d0c385 for instance 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 on hv-cinder-maste Jul 19 11:11:31.087806 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-2361c478-9150-4d58-bd1b-aa7dd60c18b7 req-2306a818-f426-4830-88ec-e8d46fb9c270 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033766 Jul 19 11:11:31.088398 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 515/1051] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:11:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:11:31.480969 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-4112c3c7-0613-4f8f-9256-ed56a67fc6d8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:31.481599 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-4112c3c7-0613-4f8f-9256-ed56a67fc6d8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:31.487343 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4112c3c7-0613-4f8f-9256-ed56a67fc6d8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:31.487782 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4112c3c7-0613-4f8f-9256-ed56a67fc6d8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:31.488532 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4112c3c7-0613-4f8f-9256-ed56a67fc6d8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:31.488987 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4112c3c7-0613-4f8f-9256-ed56a67fc6d8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:31.564546 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-aa487c85-db46-4f0b-900e-7ec618b97a6d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:31.565106 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-aa487c85-db46-4f0b-900e-7ec618b97a6d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:31.572151 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-aa487c85-db46-4f0b-900e-7ec618b97a6d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:31.572597 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-aa487c85-db46-4f0b-900e-7ec618b97a6d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:31.573382 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-aa487c85-db46-4f0b-900e-7ec618b97a6d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:31.573813 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-aa487c85-db46-4f0b-900e-7ec618b97a6d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:31.603628 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-aa487c85-db46-4f0b-900e-7ec618b97a6d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043968 Jul 19 11:11:31.604193 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 516/1052] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:11:31 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:11:31.615876 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-4112c3c7-0613-4f8f-9256-ed56a67fc6d8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:31.623794 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-4112c3c7-0613-4f8f-9256-ed56a67fc6d8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:31.625678 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-4112c3c7-0613-4f8f-9256-ed56a67fc6d8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:31.626822 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-4112c3c7-0613-4f8f-9256-ed56a67fc6d8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.149844 Jul 19 11:11:31.627371 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 537/1053] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:31 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:31.917991 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-af13ffad-eb73-43fe-be00-35b5d6d51e73 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:31.918602 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-af13ffad-eb73-43fe-be00-35b5d6d51e73 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:31.924826 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-af13ffad-eb73-43fe-be00-35b5d6d51e73 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:31.925284 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-af13ffad-eb73-43fe-be00-35b5d6d51e73 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:31.926058 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-af13ffad-eb73-43fe-be00-35b5d6d51e73 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:31.926498 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-af13ffad-eb73-43fe-be00-35b5d6d51e73 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:32.057319 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-af13ffad-eb73-43fe-be00-35b5d6d51e73 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:32.065453 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-af13ffad-eb73-43fe-be00-35b5d6d51e73 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.151363 Jul 19 11:11:32.066013 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 517/1054] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:31 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:32.618515 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-7db83598-6e71-45a1-bd4b-f0a0dd32bd3c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:32.620045 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-7db83598-6e71-45a1-bd4b-f0a0dd32bd3c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:32.627304 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7db83598-6e71-45a1-bd4b-f0a0dd32bd3c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:32.627743 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7db83598-6e71-45a1-bd4b-f0a0dd32bd3c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:32.628496 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7db83598-6e71-45a1-bd4b-f0a0dd32bd3c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:32.628961 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7db83598-6e71-45a1-bd4b-f0a0dd32bd3c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:32.644816 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-00a46a3f-ad61-4c55-808e-e5703f607147 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:32.645506 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-00a46a3f-ad61-4c55-808e-e5703f607147 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:32.651960 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-00a46a3f-ad61-4c55-808e-e5703f607147 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:32.652407 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-00a46a3f-ad61-4c55-808e-e5703f607147 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:32.653203 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-00a46a3f-ad61-4c55-808e-e5703f607147 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:32.653629 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-00a46a3f-ad61-4c55-808e-e5703f607147 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:32.658257 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-7db83598-6e71-45a1-bd4b-f0a0dd32bd3c tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045745 Jul 19 11:11:32.658859 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 538/1055] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:11:32 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:11:32.782527 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-00a46a3f-ad61-4c55-808e-e5703f607147 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:32.790435 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-00a46a3f-ad61-4c55-808e-e5703f607147 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:32.791945 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-00a46a3f-ad61-4c55-808e-e5703f607147 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:32.793135 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-00a46a3f-ad61-4c55-808e-e5703f607147 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.152295 Jul 19 11:11:32.793688 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 518/1056] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:32 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:11:33.082752 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-6bacb70c-67a1-4259-8e7a-c7241653cc60 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:33.083316 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-6bacb70c-67a1-4259-8e7a-c7241653cc60 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:33.089356 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6bacb70c-67a1-4259-8e7a-c7241653cc60 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:33.089807 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6bacb70c-67a1-4259-8e7a-c7241653cc60 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:33.090567 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6bacb70c-67a1-4259-8e7a-c7241653cc60 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:33.091004 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6bacb70c-67a1-4259-8e7a-c7241653cc60 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:33.104195 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-18298499-bff3-474b-847f-411d9ef63c5f req-eb3a81ac-7838-447b-8950-8391f7355a5e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "36abf2ae-3b4c-4095-b9ba-2bd738e266d9", "name": "network-vif-plugged", "status": "completed", "tag": "049ef62d-5f12-4134-9010-2bc20edecd41"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:11:33.112261 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-18298499-bff3-474b-847f-411d9ef63c5f req-eb3a81ac-7838-447b-8950-8391f7355a5e service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:33.112808 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-18298499-bff3-474b-847f-411d9ef63c5f req-eb3a81ac-7838-447b-8950-8391f7355a5e service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:33.123415 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-18298499-bff3-474b-847f-411d9ef63c5f req-eb3a81ac-7838-447b-8950-8391f7355a5e service nova] Creating event network-vif-plugged:049ef62d-5f12-4134-9010-2bc20edecd41 for instance 36abf2ae-3b4c-4095-b9ba-2bd738e266d9 on hv-cinder-maste Jul 19 11:11:33.128808 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-18298499-bff3-474b-847f-411d9ef63c5f req-eb3a81ac-7838-447b-8950-8391f7355a5e service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.028115 Jul 19 11:11:33.129476 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 519/1057] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:11:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:11:33.224492 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-6bacb70c-67a1-4259-8e7a-c7241653cc60 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:33.231978 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-6bacb70c-67a1-4259-8e7a-c7241653cc60 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.153744 Jul 19 11:11:33.232544 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 539/1058] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:33 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:33.670266 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-4277df66-9b17-45af-9081-80fa656d4c24 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:33.671277 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-4277df66-9b17-45af-9081-80fa656d4c24 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:33.678572 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4277df66-9b17-45af-9081-80fa656d4c24 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:33.679029 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4277df66-9b17-45af-9081-80fa656d4c24 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:33.679797 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4277df66-9b17-45af-9081-80fa656d4c24 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:33.680237 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4277df66-9b17-45af-9081-80fa656d4c24 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:33.708968 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-4277df66-9b17-45af-9081-80fa656d4c24 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043010 Jul 19 11:11:33.709516 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 520/1059] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:11:33 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:11:33.811560 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-a1065b60-61af-441f-9c34-1bb4b69fd5e7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:33.812196 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-a1065b60-61af-441f-9c34-1bb4b69fd5e7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:33.819571 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a1065b60-61af-441f-9c34-1bb4b69fd5e7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:33.820043 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a1065b60-61af-441f-9c34-1bb4b69fd5e7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:33.820829 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a1065b60-61af-441f-9c34-1bb4b69fd5e7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:33.821260 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a1065b60-61af-441f-9c34-1bb4b69fd5e7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:33.949384 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-a1065b60-61af-441f-9c34-1bb4b69fd5e7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:33.957339 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-a1065b60-61af-441f-9c34-1bb4b69fd5e7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:33.958842 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-a1065b60-61af-441f-9c34-1bb4b69fd5e7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:33.959977 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-a1065b60-61af-441f-9c34-1bb4b69fd5e7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.152305 Jul 19 11:11:33.960564 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 540/1060] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:33 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:34.248468 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-928077a2-3d13-4786-b3aa-6370ae564bd5 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:34.249024 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-928077a2-3d13-4786-b3aa-6370ae564bd5 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:34.255246 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-928077a2-3d13-4786-b3aa-6370ae564bd5 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:34.255693 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-928077a2-3d13-4786-b3aa-6370ae564bd5 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:34.256441 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-928077a2-3d13-4786-b3aa-6370ae564bd5 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:34.256914 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-928077a2-3d13-4786-b3aa-6370ae564bd5 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:34.389395 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-928077a2-3d13-4786-b3aa-6370ae564bd5 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:34.396843 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-928077a2-3d13-4786-b3aa-6370ae564bd5 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.152235 Jul 19 11:11:34.397407 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 521/1061] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:34 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:34.721097 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-0bf5a459-87fb-481f-aede-25ff1941b31d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:34.722019 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-0bf5a459-87fb-481f-aede-25ff1941b31d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:34.728025 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0bf5a459-87fb-481f-aede-25ff1941b31d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:34.728472 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0bf5a459-87fb-481f-aede-25ff1941b31d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:34.729272 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0bf5a459-87fb-481f-aede-25ff1941b31d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:34.729709 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0bf5a459-87fb-481f-aede-25ff1941b31d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:34.757704 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-0bf5a459-87fb-481f-aede-25ff1941b31d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041245 Jul 19 11:11:34.758246 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 541/1062] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:11:34 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:11:34.978015 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-b9c05cbd-b4e2-400f-aa1d-d5004f2642d1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:34.978549 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-b9c05cbd-b4e2-400f-aa1d-d5004f2642d1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:34.985032 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b9c05cbd-b4e2-400f-aa1d-d5004f2642d1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:34.986082 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b9c05cbd-b4e2-400f-aa1d-d5004f2642d1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:34.986830 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b9c05cbd-b4e2-400f-aa1d-d5004f2642d1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:34.987258 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b9c05cbd-b4e2-400f-aa1d-d5004f2642d1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:35.112607 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b9c05cbd-b4e2-400f-aa1d-d5004f2642d1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:35.120414 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b9c05cbd-b4e2-400f-aa1d-d5004f2642d1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:35.121938 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b9c05cbd-b4e2-400f-aa1d-d5004f2642d1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:35.123079 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-b9c05cbd-b4e2-400f-aa1d-d5004f2642d1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.149122 Jul 19 11:11:35.123627 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 522/1063] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:34 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:35.414061 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-6c2ad97e-b8a0-4d18-bb80-8209fa64e4e4 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:35.414591 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-6c2ad97e-b8a0-4d18-bb80-8209fa64e4e4 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:35.417022 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_api:SELECT=79,nova_api:INSERT=3 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:11:35.422320 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6c2ad97e-b8a0-4d18-bb80-8209fa64e4e4 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:35.422763 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6c2ad97e-b8a0-4d18-bb80-8209fa64e4e4 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:35.423522 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6c2ad97e-b8a0-4d18-bb80-8209fa64e4e4 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:35.423952 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6c2ad97e-b8a0-4d18-bb80-8209fa64e4e4 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:35.551596 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-6c2ad97e-b8a0-4d18-bb80-8209fa64e4e4 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:35.561519 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-6c2ad97e-b8a0-4d18-bb80-8209fa64e4e4 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.151404 Jul 19 11:11:35.562097 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 542/1064] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:35 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:35.770766 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-b686116e-ad55-41fa-b446-a6001cf9878f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:35.771718 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-b686116e-ad55-41fa-b446-a6001cf9878f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:35.777779 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b686116e-ad55-41fa-b446-a6001cf9878f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:35.778224 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b686116e-ad55-41fa-b446-a6001cf9878f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:35.778974 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b686116e-ad55-41fa-b446-a6001cf9878f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:35.779406 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b686116e-ad55-41fa-b446-a6001cf9878f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:35.806914 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-b686116e-ad55-41fa-b446-a6001cf9878f tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041478 Jul 19 11:11:35.807456 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 523/1065] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:11:35 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:11:36.140754 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-c28770dd-0e3a-443f-9f4a-9b4ec3022675 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:36.141290 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-c28770dd-0e3a-443f-9f4a-9b4ec3022675 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:36.147774 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c28770dd-0e3a-443f-9f4a-9b4ec3022675 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:36.148218 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c28770dd-0e3a-443f-9f4a-9b4ec3022675 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:36.148992 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c28770dd-0e3a-443f-9f4a-9b4ec3022675 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:36.149421 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-c28770dd-0e3a-443f-9f4a-9b4ec3022675 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:36.287531 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-c28770dd-0e3a-443f-9f4a-9b4ec3022675 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:36.295705 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-c28770dd-0e3a-443f-9f4a-9b4ec3022675 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:36.297327 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-c28770dd-0e3a-443f-9f4a-9b4ec3022675 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:36.298654 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-c28770dd-0e3a-443f-9f4a-9b4ec3022675 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.161890 Jul 19 11:11:36.299211 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 543/1066] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:36 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:36.578845 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-fdff9850-5e24-4d11-b4fd-712d69678c52 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:36.579984 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-fdff9850-5e24-4d11-b4fd-712d69678c52 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:36.586258 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fdff9850-5e24-4d11-b4fd-712d69678c52 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:36.586702 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fdff9850-5e24-4d11-b4fd-712d69678c52 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:36.587466 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fdff9850-5e24-4d11-b4fd-712d69678c52 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:36.587898 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-fdff9850-5e24-4d11-b4fd-712d69678c52 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:36.778802 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-fdff9850-5e24-4d11-b4fd-712d69678c52 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:36.786235 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-fdff9850-5e24-4d11-b4fd-712d69678c52 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.211418 Jul 19 11:11:36.786808 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 524/1067] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:36 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:11:36.819128 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-8688fac8-830a-487d-a564-74080734c278 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:36.820414 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-8688fac8-830a-487d-a564-74080734c278 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:36.827489 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8688fac8-830a-487d-a564-74080734c278 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:36.827944 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8688fac8-830a-487d-a564-74080734c278 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:36.828727 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8688fac8-830a-487d-a564-74080734c278 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:36.829174 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8688fac8-830a-487d-a564-74080734c278 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:36.864951 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-8688fac8-830a-487d-a564-74080734c278 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049788 Jul 19 11:11:36.865821 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 544/1068] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:11:36 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:11:37.249001 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-d10b3419-9b8f-4fbb-aff0-3385fa9f1de5 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:37.249608 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-d10b3419-9b8f-4fbb-aff0-3385fa9f1de5 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:37.250833 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-6340b152-0700-479f-9ce9-0cf0141c16f3 req-3cefefac-a891-4521-bd08-bbd67fe24e9f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5b72a14b-bd18-4dfa-a68f-12a5424f471c", "tag": "da843419-4401-4a26-82c2-7930b51aa1b8"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:11:37.256347 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d10b3419-9b8f-4fbb-aff0-3385fa9f1de5 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:37.256914 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d10b3419-9b8f-4fbb-aff0-3385fa9f1de5 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:37.258255 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d10b3419-9b8f-4fbb-aff0-3385fa9f1de5 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:37.258255 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d10b3419-9b8f-4fbb-aff0-3385fa9f1de5 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:37.258759 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-6340b152-0700-479f-9ce9-0cf0141c16f3 req-3cefefac-a891-4521-bd08-bbd67fe24e9f service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:37.259215 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-6340b152-0700-479f-9ce9-0cf0141c16f3 req-3cefefac-a891-4521-bd08-bbd67fe24e9f service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:37.270402 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-6340b152-0700-479f-9ce9-0cf0141c16f3 req-3cefefac-a891-4521-bd08-bbd67fe24e9f service nova] Creating event network-changed:da843419-4401-4a26-82c2-7930b51aa1b8 for instance 5b72a14b-bd18-4dfa-a68f-12a5424f471c on hv-cinder-maste Jul 19 11:11:37.277620 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-6340b152-0700-479f-9ce9-0cf0141c16f3 req-3cefefac-a891-4521-bd08-bbd67fe24e9f service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030650 Jul 19 11:11:37.278539 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 545/1069] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:11:37 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:11:37.286987 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-d10b3419-9b8f-4fbb-aff0-3385fa9f1de5 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Going to try to terminate instance {{(pid=117849) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 19 11:11:37.317385 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-7f4bcc8f-a6b6-40b0-8fb6-5e9af5a36737 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:37.317987 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-7f4bcc8f-a6b6-40b0-8fb6-5e9af5a36737 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:37.324403 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7f4bcc8f-a6b6-40b0-8fb6-5e9af5a36737 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:37.324972 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7f4bcc8f-a6b6-40b0-8fb6-5e9af5a36737 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:37.325661 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7f4bcc8f-a6b6-40b0-8fb6-5e9af5a36737 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:37.326116 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7f4bcc8f-a6b6-40b0-8fb6-5e9af5a36737 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:37.367913 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-d10b3419-9b8f-4fbb-aff0-3385fa9f1de5 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "DELETE /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 204 len: 0 microversion: 2.1 time: 0.122920 Jul 19 11:11:37.368758 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 525/1070] 10.222.0.33 () {58 vars in 1238 bytes} [Tue Jul 19 11:11:37 2022] DELETE /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 0 bytes in 125 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 19 11:11:37.378965 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-2ac9b693-3803-4d0d-bf98-62c5a63cc66c tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:37.379956 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-2ac9b693-3803-4d0d-bf98-62c5a63cc66c tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:37.387026 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2ac9b693-3803-4d0d-bf98-62c5a63cc66c tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:37.387751 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2ac9b693-3803-4d0d-bf98-62c5a63cc66c tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:37.389057 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2ac9b693-3803-4d0d-bf98-62c5a63cc66c tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:37.389646 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2ac9b693-3803-4d0d-bf98-62c5a63cc66c tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:37.474112 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-7f4bcc8f-a6b6-40b0-8fb6-5e9af5a36737 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:37.482354 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-7f4bcc8f-a6b6-40b0-8fb6-5e9af5a36737 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:37.483866 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-7f4bcc8f-a6b6-40b0-8fb6-5e9af5a36737 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:37.485074 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-7f4bcc8f-a6b6-40b0-8fb6-5e9af5a36737 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.172082 Jul 19 11:11:37.485680 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 546/1071] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:37 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:11:37.530235 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-2ac9b693-3803-4d0d-bf98-62c5a63cc66c tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:37.538718 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-2ac9b693-3803-4d0d-bf98-62c5a63cc66c tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1567 microversion: 2.1 time: 0.165326 Jul 19 11:11:37.539276 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 526/1072] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:37 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1567 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:11:37.802908 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-b41b4f0e-092e-455c-ad58-e23449ae07c1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:37.803604 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-b41b4f0e-092e-455c-ad58-e23449ae07c1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:37.809616 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b41b4f0e-092e-455c-ad58-e23449ae07c1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:37.810063 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b41b4f0e-092e-455c-ad58-e23449ae07c1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:37.810822 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b41b4f0e-092e-455c-ad58-e23449ae07c1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:37.811258 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b41b4f0e-092e-455c-ad58-e23449ae07c1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:37.880230 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-1c424ba3-5ac2-4c69-9da0-65b8f21bd2d6 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:37.881245 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-1c424ba3-5ac2-4c69-9da0-65b8f21bd2d6 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:37.893601 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1c424ba3-5ac2-4c69-9da0-65b8f21bd2d6 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:37.899151 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1c424ba3-5ac2-4c69-9da0-65b8f21bd2d6 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:37.899943 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1c424ba3-5ac2-4c69-9da0-65b8f21bd2d6 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:37.900388 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1c424ba3-5ac2-4c69-9da0-65b8f21bd2d6 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:37.939694 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-1c424ba3-5ac2-4c69-9da0-65b8f21bd2d6 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.066936 Jul 19 11:11:37.940969 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 527/1073] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:11:37 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments => generated 197 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:11:37.967085 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-b41b4f0e-092e-455c-ad58-e23449ae07c1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:37.974778 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-b41b4f0e-092e-455c-ad58-e23449ae07c1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.175715 Jul 19 11:11:37.975350 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 547/1074] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:37 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:38.503437 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-5690127d-b93a-4616-be11-dccc987b2cf3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:38.504048 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-5690127d-b93a-4616-be11-dccc987b2cf3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:38.510130 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5690127d-b93a-4616-be11-dccc987b2cf3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:38.510583 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5690127d-b93a-4616-be11-dccc987b2cf3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:38.511346 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5690127d-b93a-4616-be11-dccc987b2cf3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:38.511786 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5690127d-b93a-4616-be11-dccc987b2cf3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:38.555413 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-071152f0-571c-4bb4-ab89-1b607cbf2089 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:38.556099 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-071152f0-571c-4bb4-ab89-1b607cbf2089 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:38.562435 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-071152f0-571c-4bb4-ab89-1b607cbf2089 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:38.562884 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-071152f0-571c-4bb4-ab89-1b607cbf2089 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:38.564315 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-071152f0-571c-4bb4-ab89-1b607cbf2089 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:38.564781 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-071152f0-571c-4bb4-ab89-1b607cbf2089 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:38.643517 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-5690127d-b93a-4616-be11-dccc987b2cf3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:38.651534 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-5690127d-b93a-4616-be11-dccc987b2cf3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:38.653072 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-5690127d-b93a-4616-be11-dccc987b2cf3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:38.654252 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-5690127d-b93a-4616-be11-dccc987b2cf3 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.155574 Jul 19 11:11:38.654813 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 528/1075] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:38 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:38.695942 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-071152f0-571c-4bb4-ab89-1b607cbf2089 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:38.703296 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-071152f0-571c-4bb4-ab89-1b607cbf2089 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1567 microversion: 2.1 time: 0.151904 Jul 19 11:11:38.703855 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 548/1076] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:38 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1567 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:11:38.950617 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-75e4f0f7-3b2e-4ebe-8648-90e8a501dbe4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:38.951540 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-75e4f0f7-3b2e-4ebe-8648-90e8a501dbe4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:38.958570 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-75e4f0f7-3b2e-4ebe-8648-90e8a501dbe4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:38.959020 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-75e4f0f7-3b2e-4ebe-8648-90e8a501dbe4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:38.959777 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-75e4f0f7-3b2e-4ebe-8648-90e8a501dbe4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:38.960215 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-75e4f0f7-3b2e-4ebe-8648-90e8a501dbe4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:38.989197 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-75e4f0f7-3b2e-4ebe-8648-90e8a501dbe4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042490 Jul 19 11:11:38.989783 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 529/1077] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:11:38 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:11:38.991134 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-d34dbb39-25d8-4110-9d06-8d863ae1c4a3 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:38.991736 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-d34dbb39-25d8-4110-9d06-8d863ae1c4a3 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:38.998185 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d34dbb39-25d8-4110-9d06-8d863ae1c4a3 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:38.998627 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d34dbb39-25d8-4110-9d06-8d863ae1c4a3 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:38.999379 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d34dbb39-25d8-4110-9d06-8d863ae1c4a3 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:38.999806 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d34dbb39-25d8-4110-9d06-8d863ae1c4a3 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:39.003971 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:11:39.130752 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-d34dbb39-25d8-4110-9d06-8d863ae1c4a3 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:39.138122 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-d34dbb39-25d8-4110-9d06-8d863ae1c4a3 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.150790 Jul 19 11:11:39.138692 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 549/1078] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:38 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:11:39.484268 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-5eb4867a-8c19-4aa2-8aa5-74dd0447c0ff tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:39.484790 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-5eb4867a-8c19-4aa2-8aa5-74dd0447c0ff tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:39.490710 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5eb4867a-8c19-4aa2-8aa5-74dd0447c0ff tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:39.491156 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5eb4867a-8c19-4aa2-8aa5-74dd0447c0ff tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:39.491903 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5eb4867a-8c19-4aa2-8aa5-74dd0447c0ff tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:39.492335 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5eb4867a-8c19-4aa2-8aa5-74dd0447c0ff tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:39.514994 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-5eb4867a-8c19-4aa2-8aa5-74dd0447c0ff tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75/ips" status: 200 len: 147 microversion: 2.1 time: 0.137143 Jul 19 11:11:39.515532 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 530/1079] 10.222.0.33 () {58 vars in 1247 bytes} [Tue Jul 19 11:11:39 2022] GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75/ips => generated 147 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:11:39.672621 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-bb5a49b5-d094-4c32-bb05-723940ebd3ec tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:39.673806 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-bb5a49b5-d094-4c32-bb05-723940ebd3ec tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:39.679748 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-bb5a49b5-d094-4c32-bb05-723940ebd3ec tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:39.680209 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-bb5a49b5-d094-4c32-bb05-723940ebd3ec tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:39.681219 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-bb5a49b5-d094-4c32-bb05-723940ebd3ec tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:39.681894 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-bb5a49b5-d094-4c32-bb05-723940ebd3ec tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:39.743838 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-8ac1b8c3-5520-4784-b4bd-7d8842968d7e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:39.745075 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-8ac1b8c3-5520-4784-b4bd-7d8842968d7e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:39.753305 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8ac1b8c3-5520-4784-b4bd-7d8842968d7e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:39.753869 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8ac1b8c3-5520-4784-b4bd-7d8842968d7e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:39.754686 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8ac1b8c3-5520-4784-b4bd-7d8842968d7e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:39.755131 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-8ac1b8c3-5520-4784-b4bd-7d8842968d7e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:39.853938 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-bb5a49b5-d094-4c32-bb05-723940ebd3ec tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:39.862792 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-bb5a49b5-d094-4c32-bb05-723940ebd3ec tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:39.864310 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-bb5a49b5-d094-4c32-bb05-723940ebd3ec tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:39.865818 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-bb5a49b5-d094-4c32-bb05-723940ebd3ec tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.196748 Jul 19 11:11:39.866047 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 550/1080] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:39 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:11:39.898443 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-8ac1b8c3-5520-4784-b4bd-7d8842968d7e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:39.900762 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:11:39.907385 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-8ac1b8c3-5520-4784-b4bd-7d8842968d7e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1567 microversion: 2.1 time: 0.170742 Jul 19 11:11:39.907946 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 531/1081] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:39 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1567 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:40.000825 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-fc579294-37d3-4ac2-b826-e48503cc660e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:40.002434 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-fc579294-37d3-4ac2-b826-e48503cc660e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:40.008881 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fc579294-37d3-4ac2-b826-e48503cc660e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:40.009331 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fc579294-37d3-4ac2-b826-e48503cc660e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:40.010095 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fc579294-37d3-4ac2-b826-e48503cc660e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:40.010527 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fc579294-37d3-4ac2-b826-e48503cc660e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:40.038325 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-fc579294-37d3-4ac2-b826-e48503cc660e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041717 Jul 19 11:11:40.038865 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 551/1082] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:11:39 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:11:40.155188 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-953bd24c-2ab2-4d0c-8d5a-2717865e6e75 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:40.155709 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-953bd24c-2ab2-4d0c-8d5a-2717865e6e75 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:40.163687 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-953bd24c-2ab2-4d0c-8d5a-2717865e6e75 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:40.164127 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-953bd24c-2ab2-4d0c-8d5a-2717865e6e75 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:40.164910 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-953bd24c-2ab2-4d0c-8d5a-2717865e6e75 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:40.165343 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-953bd24c-2ab2-4d0c-8d5a-2717865e6e75 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:40.296538 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-953bd24c-2ab2-4d0c-8d5a-2717865e6e75 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:40.303958 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-953bd24c-2ab2-4d0c-8d5a-2717865e6e75 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.152704 Jul 19 11:11:40.304527 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 532/1083] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:40 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:40.883286 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-d4b4b396-91e4-4daf-87f0-b4d19d874309 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:40.883918 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-d4b4b396-91e4-4daf-87f0-b4d19d874309 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:40.889959 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d4b4b396-91e4-4daf-87f0-b4d19d874309 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:40.890407 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d4b4b396-91e4-4daf-87f0-b4d19d874309 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:40.891166 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d4b4b396-91e4-4daf-87f0-b4d19d874309 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:40.891603 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d4b4b396-91e4-4daf-87f0-b4d19d874309 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:40.924666 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-0a1a9103-b25a-4189-bb1a-081c2302855e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:40.925380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-0a1a9103-b25a-4189-bb1a-081c2302855e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:40.931569 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0a1a9103-b25a-4189-bb1a-081c2302855e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:40.934059 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0a1a9103-b25a-4189-bb1a-081c2302855e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:40.934059 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0a1a9103-b25a-4189-bb1a-081c2302855e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:40.934059 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0a1a9103-b25a-4189-bb1a-081c2302855e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:41.024424 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-d4b4b396-91e4-4daf-87f0-b4d19d874309 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:41.033249 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-d4b4b396-91e4-4daf-87f0-b4d19d874309 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:41.034813 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-d4b4b396-91e4-4daf-87f0-b4d19d874309 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:41.036011 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-d4b4b396-91e4-4daf-87f0-b4d19d874309 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.156687 Jul 19 11:11:41.036575 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 552/1084] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:40 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:41.050135 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-df21642b-4bf4-42aa-b06b-26b66a147157 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:41.050974 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-df21642b-4bf4-42aa-b06b-26b66a147157 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:41.056852 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-df21642b-4bf4-42aa-b06b-26b66a147157 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:41.057300 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-df21642b-4bf4-42aa-b06b-26b66a147157 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:41.058057 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-df21642b-4bf4-42aa-b06b-26b66a147157 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:41.058486 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-df21642b-4bf4-42aa-b06b-26b66a147157 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:41.068064 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-0a1a9103-b25a-4189-bb1a-081c2302855e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:41.075552 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-0a1a9103-b25a-4189-bb1a-081c2302855e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1567 microversion: 2.1 time: 0.155403 Jul 19 11:11:41.076112 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 533/1085] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:40 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1567 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:11:41.087303 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-df21642b-4bf4-42aa-b06b-26b66a147157 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041564 Jul 19 11:11:41.087851 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 553/1086] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:11:41 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:11:41.320510 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-2ba24f9b-1a3b-4ad2-bd49-800d054301e8 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:41.321136 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-2ba24f9b-1a3b-4ad2-bd49-800d054301e8 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:41.327636 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2ba24f9b-1a3b-4ad2-bd49-800d054301e8 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:41.328083 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2ba24f9b-1a3b-4ad2-bd49-800d054301e8 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:41.328887 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2ba24f9b-1a3b-4ad2-bd49-800d054301e8 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:41.329818 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2ba24f9b-1a3b-4ad2-bd49-800d054301e8 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:41.457159 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-2ba24f9b-1a3b-4ad2-bd49-800d054301e8 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:41.464608 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-2ba24f9b-1a3b-4ad2-bd49-800d054301e8 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.148015 Jul 19 11:11:41.465203 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 534/1087] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:41 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:41.890583 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-a71579e8-34a5-43ae-877c-14f8aec22ce6 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "d5e72a9b-47b0-49af-b136-1c4a4ae09617"}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:11:41.891674 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-a71579e8-34a5-43ae-877c-14f8aec22ce6 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:41.898333 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a71579e8-34a5-43ae-877c-14f8aec22ce6 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:41.898793 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a71579e8-34a5-43ae-877c-14f8aec22ce6 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:41.899559 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a71579e8-34a5-43ae-877c-14f8aec22ce6 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:41.899994 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a71579e8-34a5-43ae-877c-14f8aec22ce6 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:42.054286 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-b8abccc4-af73-4f7a-af59-7384031dae31 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:42.054969 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-b8abccc4-af73-4f7a-af59-7384031dae31 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:42.061465 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b8abccc4-af73-4f7a-af59-7384031dae31 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:42.061996 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b8abccc4-af73-4f7a-af59-7384031dae31 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:42.062774 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b8abccc4-af73-4f7a-af59-7384031dae31 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:42.063221 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b8abccc4-af73-4f7a-af59-7384031dae31 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:42.198427 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b8abccc4-af73-4f7a-af59-7384031dae31 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:42.206367 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b8abccc4-af73-4f7a-af59-7384031dae31 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:42.207879 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b8abccc4-af73-4f7a-af59-7384031dae31 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:42.209042 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-b8abccc4-af73-4f7a-af59-7384031dae31 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.158577 Jul 19 11:11:42.209598 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 535/1088] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:42 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:11:42.213874 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-96061c18-c4fd-4cd4-b4eb-0a4ccb55ed2b tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:42.214468 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-96061c18-c4fd-4cd4-b4eb-0a4ccb55ed2b tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:42.220070 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-96061c18-c4fd-4cd4-b4eb-0a4ccb55ed2b tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:42.220523 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-96061c18-c4fd-4cd4-b4eb-0a4ccb55ed2b tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:42.221323 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-96061c18-c4fd-4cd4-b4eb-0a4ccb55ed2b tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:42.221758 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-96061c18-c4fd-4cd4-b4eb-0a4ccb55ed2b tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:42.348804 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-96061c18-c4fd-4cd4-b4eb-0a4ccb55ed2b tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:42.356195 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-96061c18-c4fd-4cd4-b4eb-0a4ccb55ed2b tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1567 microversion: 2.1 time: 0.145670 Jul 19 11:11:42.356789 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 536/1089] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:42 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1567 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:42.361969 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-d274a14a-78c3-4e45-8347-3fc9982b6be0 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:42.362854 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-d274a14a-78c3-4e45-8347-3fc9982b6be0 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:42.369248 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d274a14a-78c3-4e45-8347-3fc9982b6be0 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:42.369692 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d274a14a-78c3-4e45-8347-3fc9982b6be0 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:42.370452 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d274a14a-78c3-4e45-8347-3fc9982b6be0 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:42.370884 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d274a14a-78c3-4e45-8347-3fc9982b6be0 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:42.399648 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-d274a14a-78c3-4e45-8347-3fc9982b6be0 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041872 Jul 19 11:11:42.400191 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 537/1090] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:11:42 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:11:42.480549 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-2a602835-8adb-4a4b-a2ec-f97436e50178 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:42.481172 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-2a602835-8adb-4a4b-a2ec-f97436e50178 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:42.487541 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2a602835-8adb-4a4b-a2ec-f97436e50178 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:42.487986 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2a602835-8adb-4a4b-a2ec-f97436e50178 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:42.488765 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2a602835-8adb-4a4b-a2ec-f97436e50178 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:42.489195 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-2a602835-8adb-4a4b-a2ec-f97436e50178 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:42.622328 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-2a602835-8adb-4a4b-a2ec-f97436e50178 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:42.630426 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-2a602835-8adb-4a4b-a2ec-f97436e50178 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.153586 Jul 19 11:11:42.630978 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 538/1091] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:42 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:43.227016 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-d114e7fe-c044-41a0-8550-ab2088be74bf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:43.227614 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-d114e7fe-c044-41a0-8550-ab2088be74bf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:43.229667 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_api:SELECT=91,nova_api:UPDATE=1 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:11:43.235070 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d114e7fe-c044-41a0-8550-ab2088be74bf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:43.235521 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d114e7fe-c044-41a0-8550-ab2088be74bf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:43.236283 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d114e7fe-c044-41a0-8550-ab2088be74bf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:43.236728 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d114e7fe-c044-41a0-8550-ab2088be74bf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:43.371211 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-d114e7fe-c044-41a0-8550-ab2088be74bf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:43.380016 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-d114e7fe-c044-41a0-8550-ab2088be74bf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:43.381585 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-d114e7fe-c044-41a0-8550-ab2088be74bf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:43.382734 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-d114e7fe-c044-41a0-8550-ab2088be74bf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.159575 Jul 19 11:11:43.383289 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 539/1092] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:43 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:11:43.387630 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-ceef9918-21ed-4dc5-9451-abb57a825136 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:43.388228 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-ceef9918-21ed-4dc5-9451-abb57a825136 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:43.394631 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ceef9918-21ed-4dc5-9451-abb57a825136 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:43.395086 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ceef9918-21ed-4dc5-9451-abb57a825136 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:43.395851 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ceef9918-21ed-4dc5-9451-abb57a825136 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:43.396297 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ceef9918-21ed-4dc5-9451-abb57a825136 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:43.530350 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-ceef9918-21ed-4dc5-9451-abb57a825136 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:43.538363 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-ceef9918-21ed-4dc5-9451-abb57a825136 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1567 microversion: 2.1 time: 0.154130 Jul 19 11:11:43.538943 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 540/1093] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:43 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1567 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:43.543506 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-a13d82ea-116a-4e9a-995a-db6686b199d3 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:43.544470 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-a13d82ea-116a-4e9a-995a-db6686b199d3 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:43.552538 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a13d82ea-116a-4e9a-995a-db6686b199d3 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:43.553049 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a13d82ea-116a-4e9a-995a-db6686b199d3 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:43.553828 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a13d82ea-116a-4e9a-995a-db6686b199d3 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:43.554267 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a13d82ea-116a-4e9a-995a-db6686b199d3 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:43.583872 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-a13d82ea-116a-4e9a-995a-db6686b199d3 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043903 Jul 19 11:11:43.584434 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 541/1094] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:11:43 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:11:43.646804 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-54eef032-49ca-465d-b922-ea49ed56555a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:43.647406 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-54eef032-49ca-465d-b922-ea49ed56555a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:43.653376 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-54eef032-49ca-465d-b922-ea49ed56555a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:43.653831 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-54eef032-49ca-465d-b922-ea49ed56555a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:43.654592 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-54eef032-49ca-465d-b922-ea49ed56555a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:43.655038 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-54eef032-49ca-465d-b922-ea49ed56555a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:43.790439 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-54eef032-49ca-465d-b922-ea49ed56555a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:43.798402 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-54eef032-49ca-465d-b922-ea49ed56555a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.154817 Jul 19 11:11:43.798971 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 542/1095] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:43 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:11:44.402119 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-9e820663-d160-4dac-a41b-207a0e99d980 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:44.402763 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-9e820663-d160-4dac-a41b-207a0e99d980 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:44.409914 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9e820663-d160-4dac-a41b-207a0e99d980 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:44.410371 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9e820663-d160-4dac-a41b-207a0e99d980 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:44.411147 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9e820663-d160-4dac-a41b-207a0e99d980 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:44.411586 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9e820663-d160-4dac-a41b-207a0e99d980 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:44.545098 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-9e820663-d160-4dac-a41b-207a0e99d980 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:44.553095 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-9e820663-d160-4dac-a41b-207a0e99d980 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:44.554600 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-9e820663-d160-4dac-a41b-207a0e99d980 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:44.555719 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-9e820663-d160-4dac-a41b-207a0e99d980 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.157714 Jul 19 11:11:44.556268 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 543/1096] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:44 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:11:44.560597 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-a25d7d50-2df6-4c86-9835-8be40099d67a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:44.561228 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-a25d7d50-2df6-4c86-9835-8be40099d67a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:44.567364 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a25d7d50-2df6-4c86-9835-8be40099d67a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:44.567811 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a25d7d50-2df6-4c86-9835-8be40099d67a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:44.568575 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a25d7d50-2df6-4c86-9835-8be40099d67a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:44.569031 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a25d7d50-2df6-4c86-9835-8be40099d67a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:44.699511 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-a25d7d50-2df6-4c86-9835-8be40099d67a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:44.707648 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-a25d7d50-2df6-4c86-9835-8be40099d67a tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1567 microversion: 2.1 time: 0.150432 Jul 19 11:11:44.708270 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 544/1097] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:44 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1567 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:44.712988 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-a9b4e5f0-d4c5-4846-ac3c-4db66ad2f4e8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:44.713954 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-a9b4e5f0-d4c5-4846-ac3c-4db66ad2f4e8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:44.719624 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a9b4e5f0-d4c5-4846-ac3c-4db66ad2f4e8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:44.720069 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a9b4e5f0-d4c5-4846-ac3c-4db66ad2f4e8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:44.720852 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a9b4e5f0-d4c5-4846-ac3c-4db66ad2f4e8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:44.721280 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a9b4e5f0-d4c5-4846-ac3c-4db66ad2f4e8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:44.750467 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-a9b4e5f0-d4c5-4846-ac3c-4db66ad2f4e8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040871 Jul 19 11:11:44.751515 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 545/1098] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:11:44 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:11:44.815399 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-e8e0309c-8220-44c6-a976-59f4f2d18ba0 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:44.815932 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-e8e0309c-8220-44c6-a976-59f4f2d18ba0 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:44.821934 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e8e0309c-8220-44c6-a976-59f4f2d18ba0 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:44.822388 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e8e0309c-8220-44c6-a976-59f4f2d18ba0 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:44.823141 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e8e0309c-8220-44c6-a976-59f4f2d18ba0 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:44.823580 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e8e0309c-8220-44c6-a976-59f4f2d18ba0 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:44.956295 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-e8e0309c-8220-44c6-a976-59f4f2d18ba0 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:44.963645 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-e8e0309c-8220-44c6-a976-59f4f2d18ba0 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.152678 Jul 19 11:11:44.964195 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 546/1099] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:44 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:11:45.512290 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-a71579e8-34a5-43ae-877c-14f8aec22ce6 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "POST /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 3.625682 Jul 19 11:11:45.512933 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 554/1100] 10.222.0.33 () {60 vars in 1322 bytes} [Tue Jul 19 11:11:41 2022] POST /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75/os-volume_attachments => generated 194 bytes in 3627 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:11:45.575673 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-4419824e-82b6-4b80-8af4-2fc56f3f34c5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:45.576290 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-4419824e-82b6-4b80-8af4-2fc56f3f34c5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:45.583866 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4419824e-82b6-4b80-8af4-2fc56f3f34c5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:45.584323 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4419824e-82b6-4b80-8af4-2fc56f3f34c5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:45.585142 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4419824e-82b6-4b80-8af4-2fc56f3f34c5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:45.585584 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4419824e-82b6-4b80-8af4-2fc56f3f34c5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:45.717811 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-4419824e-82b6-4b80-8af4-2fc56f3f34c5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:45.726014 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-3887015c-722f-4908-9f48-7f052ffc516f tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:45.726620 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-4419824e-82b6-4b80-8af4-2fc56f3f34c5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:45.726948 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-3887015c-722f-4908-9f48-7f052ffc516f tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:45.728164 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-4419824e-82b6-4b80-8af4-2fc56f3f34c5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:45.729314 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-4419824e-82b6-4b80-8af4-2fc56f3f34c5 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.158840 Jul 19 11:11:45.730591 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 547/1101] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:45 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:45.734048 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3887015c-722f-4908-9f48-7f052ffc516f tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:45.734606 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3887015c-722f-4908-9f48-7f052ffc516f tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:45.735955 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3887015c-722f-4908-9f48-7f052ffc516f tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:45.736461 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3887015c-722f-4908-9f48-7f052ffc516f tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:45.762709 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-ee02303e-0f12-4852-9d8a-6f7f7c49c2b4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:45.763677 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-ee02303e-0f12-4852-9d8a-6f7f7c49c2b4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:45.769841 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ee02303e-0f12-4852-9d8a-6f7f7c49c2b4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:45.770285 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ee02303e-0f12-4852-9d8a-6f7f7c49c2b4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:45.771053 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ee02303e-0f12-4852-9d8a-6f7f7c49c2b4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:45.771559 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ee02303e-0f12-4852-9d8a-6f7f7c49c2b4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:45.785428 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=100 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:11:45.801875 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-ee02303e-0f12-4852-9d8a-6f7f7c49c2b4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042978 Jul 19 11:11:45.802452 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 548/1102] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:11:45 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:11:45.878915 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-3887015c-722f-4908-9f48-7f052ffc516f tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:45.886504 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-3887015c-722f-4908-9f48-7f052ffc516f tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1567 microversion: 2.1 time: 0.165182 Jul 19 11:11:45.887086 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 555/1103] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:45 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1567 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:11:45.981054 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-c9db88e8-fd86-43f5-b5da-049406104d05 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:45.981589 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-c9db88e8-fd86-43f5-b5da-049406104d05 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:45.987978 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c9db88e8-fd86-43f5-b5da-049406104d05 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:45.988426 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c9db88e8-fd86-43f5-b5da-049406104d05 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:45.989192 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c9db88e8-fd86-43f5-b5da-049406104d05 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:45.989620 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c9db88e8-fd86-43f5-b5da-049406104d05 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:46.115215 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-c9db88e8-fd86-43f5-b5da-049406104d05 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:46.122673 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-c9db88e8-fd86-43f5-b5da-049406104d05 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.145594 Jul 19 11:11:46.123232 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 549/1104] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:45 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:46.747432 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-1056bdf8-fc3a-4e6a-bf83-409cc7b914d1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:46.748063 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-1056bdf8-fc3a-4e6a-bf83-409cc7b914d1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:46.755023 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1056bdf8-fc3a-4e6a-bf83-409cc7b914d1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:46.755477 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1056bdf8-fc3a-4e6a-bf83-409cc7b914d1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:46.756235 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1056bdf8-fc3a-4e6a-bf83-409cc7b914d1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:46.756669 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1056bdf8-fc3a-4e6a-bf83-409cc7b914d1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:46.813016 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-aa730c80-185d-4034-b5ff-2e34fa1e17c6 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:46.814453 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-aa730c80-185d-4034-b5ff-2e34fa1e17c6 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:46.821642 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-aa730c80-185d-4034-b5ff-2e34fa1e17c6 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:46.822091 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-aa730c80-185d-4034-b5ff-2e34fa1e17c6 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:46.822875 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-aa730c80-185d-4034-b5ff-2e34fa1e17c6 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:46.823311 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-aa730c80-185d-4034-b5ff-2e34fa1e17c6 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:46.854024 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-aa730c80-185d-4034-b5ff-2e34fa1e17c6 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044804 Jul 19 11:11:46.855045 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 550/1105] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:11:46 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:11:46.895549 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-1056bdf8-fc3a-4e6a-bf83-409cc7b914d1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:46.903490 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-33c6c332-f94a-493b-ada6-d2a0d35c6bd1 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:46.904099 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-33c6c332-f94a-493b-ada6-d2a0d35c6bd1 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:46.904808 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-1056bdf8-fc3a-4e6a-bf83-409cc7b914d1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:46.906959 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-1056bdf8-fc3a-4e6a-bf83-409cc7b914d1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:46.908281 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-1056bdf8-fc3a-4e6a-bf83-409cc7b914d1 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.164918 Jul 19 11:11:46.909418 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 556/1106] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:46 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:46.922632 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-33c6c332-f94a-493b-ada6-d2a0d35c6bd1 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:46.923217 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-33c6c332-f94a-493b-ada6-d2a0d35c6bd1 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:46.924032 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-33c6c332-f94a-493b-ada6-d2a0d35c6bd1 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:46.924468 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-33c6c332-f94a-493b-ada6-d2a0d35c6bd1 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:47.056254 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-33c6c332-f94a-493b-ada6-d2a0d35c6bd1 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:47.063953 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-33c6c332-f94a-493b-ada6-d2a0d35c6bd1 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1567 microversion: 2.1 time: 0.164477 Jul 19 11:11:47.064891 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 551/1107] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:46 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1567 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:11:47.139453 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-67bf1b60-d59c-479f-a22e-276b580ef0ca tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:47.140181 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-67bf1b60-d59c-479f-a22e-276b580ef0ca tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:47.146146 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-67bf1b60-d59c-479f-a22e-276b580ef0ca tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:47.146599 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-67bf1b60-d59c-479f-a22e-276b580ef0ca tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:47.147376 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-67bf1b60-d59c-479f-a22e-276b580ef0ca tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:47.147789 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-67bf1b60-d59c-479f-a22e-276b580ef0ca tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:47.279981 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-67bf1b60-d59c-479f-a22e-276b580ef0ca tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:47.288532 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-67bf1b60-d59c-479f-a22e-276b580ef0ca tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.152926 Jul 19 11:11:47.289194 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 557/1108] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:47 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:47.866861 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-71fe1cde-c889-4f5d-ac2b-f4887657ae02 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:47.867824 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-71fe1cde-c889-4f5d-ac2b-f4887657ae02 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:47.874458 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-71fe1cde-c889-4f5d-ac2b-f4887657ae02 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:47.875145 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-71fe1cde-c889-4f5d-ac2b-f4887657ae02 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:47.876106 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-71fe1cde-c889-4f5d-ac2b-f4887657ae02 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:47.876545 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-71fe1cde-c889-4f5d-ac2b-f4887657ae02 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:47.904944 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-71fe1cde-c889-4f5d-ac2b-f4887657ae02 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041831 Jul 19 11:11:47.905715 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 552/1109] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:11:47 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:11:47.927060 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-ca0b1bf6-3e6b-422a-a7b0-129523663902 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:47.927658 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-ca0b1bf6-3e6b-422a-a7b0-129523663902 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:47.933768 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ca0b1bf6-3e6b-422a-a7b0-129523663902 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:47.934213 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ca0b1bf6-3e6b-422a-a7b0-129523663902 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:47.934973 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ca0b1bf6-3e6b-422a-a7b0-129523663902 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:47.935408 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ca0b1bf6-3e6b-422a-a7b0-129523663902 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:48.071844 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-ca0b1bf6-3e6b-422a-a7b0-129523663902 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:48.080221 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-ca0b1bf6-3e6b-422a-a7b0-129523663902 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:48.080859 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-afc0c2b7-d456-48ef-8edb-868879ba0370 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:48.081559 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-afc0c2b7-d456-48ef-8edb-868879ba0370 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:48.081799 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-ca0b1bf6-3e6b-422a-a7b0-129523663902 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:48.082950 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-ca0b1bf6-3e6b-422a-a7b0-129523663902 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.160372 Jul 19 11:11:48.083767 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 558/1110] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:47 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:48.094353 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-afc0c2b7-d456-48ef-8edb-868879ba0370 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:48.094846 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-afc0c2b7-d456-48ef-8edb-868879ba0370 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:48.095614 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-afc0c2b7-d456-48ef-8edb-868879ba0370 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:48.096046 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-afc0c2b7-d456-48ef-8edb-868879ba0370 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:48.229332 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-afc0c2b7-d456-48ef-8edb-868879ba0370 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:48.237209 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-afc0c2b7-d456-48ef-8edb-868879ba0370 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1567 microversion: 2.1 time: 0.160285 Jul 19 11:11:48.237772 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 553/1111] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:48 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1567 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:11:48.304894 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-b1c372b1-00b8-49d4-a29b-2bf5fdca7b74 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:48.305633 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-b1c372b1-00b8-49d4-a29b-2bf5fdca7b74 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:48.312049 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b1c372b1-00b8-49d4-a29b-2bf5fdca7b74 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:48.312478 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b1c372b1-00b8-49d4-a29b-2bf5fdca7b74 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:48.313278 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b1c372b1-00b8-49d4-a29b-2bf5fdca7b74 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:48.314229 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b1c372b1-00b8-49d4-a29b-2bf5fdca7b74 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:48.468219 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-b1c372b1-00b8-49d4-a29b-2bf5fdca7b74 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:48.476214 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-b1c372b1-00b8-49d4-a29b-2bf5fdca7b74 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.175124 Jul 19 11:11:48.476808 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 559/1112] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:48 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:48.916336 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-be965ebc-3775-474a-965a-7007c75c44e2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:48.917294 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-be965ebc-3775-474a-965a-7007c75c44e2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:48.923781 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-be965ebc-3775-474a-965a-7007c75c44e2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:48.924224 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-be965ebc-3775-474a-965a-7007c75c44e2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:48.925015 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-be965ebc-3775-474a-965a-7007c75c44e2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:48.925447 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-be965ebc-3775-474a-965a-7007c75c44e2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:48.955306 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-be965ebc-3775-474a-965a-7007c75c44e2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042976 Jul 19 11:11:48.955853 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 554/1113] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:11:48 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:11:49.100893 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-374c8b61-a8da-4f22-9b66-ab1acf8c1d4c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:49.101526 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-374c8b61-a8da-4f22-9b66-ab1acf8c1d4c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:49.107480 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-374c8b61-a8da-4f22-9b66-ab1acf8c1d4c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:49.107923 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-374c8b61-a8da-4f22-9b66-ab1acf8c1d4c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:49.108671 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-374c8b61-a8da-4f22-9b66-ab1acf8c1d4c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:49.109124 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-374c8b61-a8da-4f22-9b66-ab1acf8c1d4c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:49.234462 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-374c8b61-a8da-4f22-9b66-ab1acf8c1d4c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:49.242356 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-374c8b61-a8da-4f22-9b66-ab1acf8c1d4c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:49.243867 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-374c8b61-a8da-4f22-9b66-ab1acf8c1d4c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:49.245043 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-374c8b61-a8da-4f22-9b66-ab1acf8c1d4c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.148040 Jul 19 11:11:49.245600 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 560/1114] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:49 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:49.255210 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-faa1058e-edbe-4b59-b5c6-fc153d9422b7 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:49.255809 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-faa1058e-edbe-4b59-b5c6-fc153d9422b7 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:49.261796 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-faa1058e-edbe-4b59-b5c6-fc153d9422b7 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:49.262236 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-faa1058e-edbe-4b59-b5c6-fc153d9422b7 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:49.262981 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-faa1058e-edbe-4b59-b5c6-fc153d9422b7 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:49.263408 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-faa1058e-edbe-4b59-b5c6-fc153d9422b7 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:49.393459 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-faa1058e-edbe-4b59-b5c6-fc153d9422b7 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:49.400962 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-faa1058e-edbe-4b59-b5c6-fc153d9422b7 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1567 microversion: 2.1 time: 0.149983 Jul 19 11:11:49.401569 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 555/1115] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:49 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1567 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:49.492389 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-6a4e1440-f3ef-4e8c-b911-b8d4d4586ad6 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:49.492974 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-6a4e1440-f3ef-4e8c-b911-b8d4d4586ad6 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:49.499828 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6a4e1440-f3ef-4e8c-b911-b8d4d4586ad6 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:49.500613 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6a4e1440-f3ef-4e8c-b911-b8d4d4586ad6 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:49.502000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6a4e1440-f3ef-4e8c-b911-b8d4d4586ad6 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:49.502431 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-6a4e1440-f3ef-4e8c-b911-b8d4d4586ad6 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:49.646182 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-6a4e1440-f3ef-4e8c-b911-b8d4d4586ad6 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:49.653866 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-6a4e1440-f3ef-4e8c-b911-b8d4d4586ad6 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.165292 Jul 19 11:11:49.654427 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 561/1116] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:49 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:49.966837 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-d2806e6d-4295-4cd6-8350-d60e4d1eefa8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:49.967794 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-d2806e6d-4295-4cd6-8350-d60e4d1eefa8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:49.974816 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d2806e6d-4295-4cd6-8350-d60e4d1eefa8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:49.975265 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d2806e6d-4295-4cd6-8350-d60e4d1eefa8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:49.976023 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d2806e6d-4295-4cd6-8350-d60e4d1eefa8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:49.976457 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d2806e6d-4295-4cd6-8350-d60e4d1eefa8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:50.005379 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-d2806e6d-4295-4cd6-8350-d60e4d1eefa8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042483 Jul 19 11:11:50.005967 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 556/1117] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:11:49 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:11:50.263573 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-55b875c8-c2b5-4302-a7ee-1f6b95988ad7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:50.264110 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-55b875c8-c2b5-4302-a7ee-1f6b95988ad7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:50.270618 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-55b875c8-c2b5-4302-a7ee-1f6b95988ad7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:50.271064 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-55b875c8-c2b5-4302-a7ee-1f6b95988ad7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:50.271808 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-55b875c8-c2b5-4302-a7ee-1f6b95988ad7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:50.272236 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-55b875c8-c2b5-4302-a7ee-1f6b95988ad7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:50.398649 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-55b875c8-c2b5-4302-a7ee-1f6b95988ad7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:50.406780 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-55b875c8-c2b5-4302-a7ee-1f6b95988ad7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:50.408949 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-55b875c8-c2b5-4302-a7ee-1f6b95988ad7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:50.410101 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-55b875c8-c2b5-4302-a7ee-1f6b95988ad7 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.150432 Jul 19 11:11:50.410639 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 562/1118] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:50 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:50.417709 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-aa0e3639-1aef-490a-938a-4e8b39300e08 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:50.418318 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-aa0e3639-1aef-490a-938a-4e8b39300e08 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:50.424270 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-aa0e3639-1aef-490a-938a-4e8b39300e08 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:50.424754 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-aa0e3639-1aef-490a-938a-4e8b39300e08 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:50.425521 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-aa0e3639-1aef-490a-938a-4e8b39300e08 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:50.425958 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-aa0e3639-1aef-490a-938a-4e8b39300e08 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:50.553853 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-aa0e3639-1aef-490a-938a-4e8b39300e08 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:50.561455 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-aa0e3639-1aef-490a-938a-4e8b39300e08 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1567 microversion: 2.1 time: 0.147517 Jul 19 11:11:50.562030 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 557/1119] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:50 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1567 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:50.670692 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-114d4e4a-63e8-4ffb-9bbf-67a8c9922550 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:50.671358 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-114d4e4a-63e8-4ffb-9bbf-67a8c9922550 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:50.678893 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-114d4e4a-63e8-4ffb-9bbf-67a8c9922550 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:50.679340 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-114d4e4a-63e8-4ffb-9bbf-67a8c9922550 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:50.680095 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-114d4e4a-63e8-4ffb-9bbf-67a8c9922550 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:50.680525 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-114d4e4a-63e8-4ffb-9bbf-67a8c9922550 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:50.812629 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-114d4e4a-63e8-4ffb-9bbf-67a8c9922550 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:50.820145 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-114d4e4a-63e8-4ffb-9bbf-67a8c9922550 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.153445 Jul 19 11:11:50.821024 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 563/1120] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:50 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:50.963890 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-d749a118-b176-4a72-938e-055ffbcf8a2c req-5c544406-b91f-48b1-bf8a-b3e5e9c02418 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5b72a14b-bd18-4dfa-a68f-12a5424f471c", "name": "network-vif-unplugged", "status": "completed", "tag": "da843419-4401-4a26-82c2-7930b51aa1b8"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:11:50.974329 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-d749a118-b176-4a72-938e-055ffbcf8a2c req-5c544406-b91f-48b1-bf8a-b3e5e9c02418 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:50.974955 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-d749a118-b176-4a72-938e-055ffbcf8a2c req-5c544406-b91f-48b1-bf8a-b3e5e9c02418 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:50.986231 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-d749a118-b176-4a72-938e-055ffbcf8a2c req-5c544406-b91f-48b1-bf8a-b3e5e9c02418 service nova] Creating event network-vif-unplugged:da843419-4401-4a26-82c2-7930b51aa1b8 for instance 5b72a14b-bd18-4dfa-a68f-12a5424f471c on hv-cinder-maste Jul 19 11:11:50.991129 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-d749a118-b176-4a72-938e-055ffbcf8a2c req-5c544406-b91f-48b1-bf8a-b3e5e9c02418 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.030907 Jul 19 11:11:50.991713 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 558/1121] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:11:50 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:11:51.017579 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-65184dad-ccf6-49c9-bb94-747171e86cea tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:51.018644 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-65184dad-ccf6-49c9-bb94-747171e86cea tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:51.024689 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-65184dad-ccf6-49c9-bb94-747171e86cea tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:51.025244 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-65184dad-ccf6-49c9-bb94-747171e86cea tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:51.026161 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-65184dad-ccf6-49c9-bb94-747171e86cea tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:51.026760 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-65184dad-ccf6-49c9-bb94-747171e86cea tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:51.044286 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:11:51.058999 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-65184dad-ccf6-49c9-bb94-747171e86cea tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045875 Jul 19 11:11:51.059494 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 564/1122] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:11:51 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:11:51.428662 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-66f8014e-f276-49b4-ab7a-70b0504ef036 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:51.429383 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-66f8014e-f276-49b4-ab7a-70b0504ef036 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:51.435623 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-66f8014e-f276-49b4-ab7a-70b0504ef036 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:51.436076 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-66f8014e-f276-49b4-ab7a-70b0504ef036 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:51.436862 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-66f8014e-f276-49b4-ab7a-70b0504ef036 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:51.437297 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-66f8014e-f276-49b4-ab7a-70b0504ef036 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:51.567158 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-66f8014e-f276-49b4-ab7a-70b0504ef036 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:51.575042 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-66f8014e-f276-49b4-ab7a-70b0504ef036 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:51.576554 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-66f8014e-f276-49b4-ab7a-70b0504ef036 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:51.577780 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-66f8014e-f276-49b4-ab7a-70b0504ef036 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.153482 Jul 19 11:11:51.578336 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 559/1123] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:51 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:11:51.580077 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-70351fee-8b27-4b16-bd59-2b5baa1cd34c tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:51.580847 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-70351fee-8b27-4b16-bd59-2b5baa1cd34c tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:51.586908 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-70351fee-8b27-4b16-bd59-2b5baa1cd34c tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:51.587355 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-70351fee-8b27-4b16-bd59-2b5baa1cd34c tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:51.588144 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-70351fee-8b27-4b16-bd59-2b5baa1cd34c tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:51.588572 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-70351fee-8b27-4b16-bd59-2b5baa1cd34c tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:51.723625 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-70351fee-8b27-4b16-bd59-2b5baa1cd34c tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:51.731141 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-70351fee-8b27-4b16-bd59-2b5baa1cd34c tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1567 microversion: 2.1 time: 0.155694 Jul 19 11:11:51.731700 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 565/1124] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:51 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1567 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:11:51.836799 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-1fe5129b-93e4-49cc-ba1a-2906b02afe62 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:51.837409 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-1fe5129b-93e4-49cc-ba1a-2906b02afe62 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:51.843541 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1fe5129b-93e4-49cc-ba1a-2906b02afe62 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:51.844000 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1fe5129b-93e4-49cc-ba1a-2906b02afe62 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:51.844798 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1fe5129b-93e4-49cc-ba1a-2906b02afe62 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:51.845334 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1fe5129b-93e4-49cc-ba1a-2906b02afe62 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:51.975136 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-1fe5129b-93e4-49cc-ba1a-2906b02afe62 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:51.983002 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-1fe5129b-93e4-49cc-ba1a-2906b02afe62 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.150109 Jul 19 11:11:51.983572 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 560/1125] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:51 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:52.070933 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-4bfa6d07-3c6c-458f-bba4-dd7eb897dd17 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:52.071907 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-4bfa6d07-3c6c-458f-bba4-dd7eb897dd17 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:52.078068 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4bfa6d07-3c6c-458f-bba4-dd7eb897dd17 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:52.078515 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4bfa6d07-3c6c-458f-bba4-dd7eb897dd17 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:52.079286 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4bfa6d07-3c6c-458f-bba4-dd7eb897dd17 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:52.079718 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4bfa6d07-3c6c-458f-bba4-dd7eb897dd17 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:52.108191 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-4bfa6d07-3c6c-458f-bba4-dd7eb897dd17 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041343 Jul 19 11:11:52.108750 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 566/1126] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:11:52 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:11:52.596080 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-a07fc3c7-8cfd-494b-9055-21dee6f089cf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:52.596687 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-a07fc3c7-8cfd-494b-9055-21dee6f089cf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:52.603079 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a07fc3c7-8cfd-494b-9055-21dee6f089cf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:52.603542 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a07fc3c7-8cfd-494b-9055-21dee6f089cf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:52.604315 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a07fc3c7-8cfd-494b-9055-21dee6f089cf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:52.604775 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a07fc3c7-8cfd-494b-9055-21dee6f089cf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:52.730518 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-a07fc3c7-8cfd-494b-9055-21dee6f089cf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:52.738496 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-a07fc3c7-8cfd-494b-9055-21dee6f089cf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:52.740048 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-a07fc3c7-8cfd-494b-9055-21dee6f089cf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:52.741222 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-a07fc3c7-8cfd-494b-9055-21dee6f089cf tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.149736 Jul 19 11:11:52.741771 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 561/1127] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:52 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:52.747598 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-36449717-a43a-46bf-9de3-b65dacd39557 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:52.748226 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-36449717-a43a-46bf-9de3-b65dacd39557 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:52.754978 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-36449717-a43a-46bf-9de3-b65dacd39557 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:52.755425 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-36449717-a43a-46bf-9de3-b65dacd39557 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:52.756218 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-36449717-a43a-46bf-9de3-b65dacd39557 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:52.756654 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-36449717-a43a-46bf-9de3-b65dacd39557 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:52.885573 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-36449717-a43a-46bf-9de3-b65dacd39557 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:52.893682 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-36449717-a43a-46bf-9de3-b65dacd39557 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1567 microversion: 2.1 time: 0.149877 Jul 19 11:11:52.894255 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 567/1128] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:52 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1567 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:52.999221 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-12688433-7ed7-45dd-ab04-06214f6ff6d0 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:52.999832 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-12688433-7ed7-45dd-ab04-06214f6ff6d0 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:53.006628 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-12688433-7ed7-45dd-ab04-06214f6ff6d0 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:53.007092 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-12688433-7ed7-45dd-ab04-06214f6ff6d0 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:53.007854 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-12688433-7ed7-45dd-ab04-06214f6ff6d0 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:53.008295 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-12688433-7ed7-45dd-ab04-06214f6ff6d0 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:53.119460 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-d240a94f-8007-43ba-aa4e-667ef330e8d3 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:53.120397 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-d240a94f-8007-43ba-aa4e-667ef330e8d3 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:53.127082 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d240a94f-8007-43ba-aa4e-667ef330e8d3 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:53.127532 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d240a94f-8007-43ba-aa4e-667ef330e8d3 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:53.128299 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d240a94f-8007-43ba-aa4e-667ef330e8d3 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:53.128830 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d240a94f-8007-43ba-aa4e-667ef330e8d3 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:53.137443 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-12688433-7ed7-45dd-ab04-06214f6ff6d0 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:53.145527 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-12688433-7ed7-45dd-ab04-06214f6ff6d0 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.150137 Jul 19 11:11:53.146111 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 562/1129] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:52 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:53.158445 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-d240a94f-8007-43ba-aa4e-667ef330e8d3 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042914 Jul 19 11:11:53.158987 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 568/1130] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:11:53 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:11:53.760232 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-03ccfc21-887f-4dcd-a892-f29b532bc390 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:53.760880 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-03ccfc21-887f-4dcd-a892-f29b532bc390 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:53.767492 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-03ccfc21-887f-4dcd-a892-f29b532bc390 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:53.767945 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-03ccfc21-887f-4dcd-a892-f29b532bc390 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:53.768748 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-03ccfc21-887f-4dcd-a892-f29b532bc390 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:53.769472 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-03ccfc21-887f-4dcd-a892-f29b532bc390 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:53.897230 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-03ccfc21-887f-4dcd-a892-f29b532bc390 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:53.905113 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-03ccfc21-887f-4dcd-a892-f29b532bc390 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:53.906633 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-03ccfc21-887f-4dcd-a892-f29b532bc390 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:53.908268 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-03ccfc21-887f-4dcd-a892-f29b532bc390 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.152819 Jul 19 11:11:53.908841 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 563/1131] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:53 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:53.911146 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-f9afa34a-ffba-4ba5-b03a-99547b1fd72d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:53.911757 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-f9afa34a-ffba-4ba5-b03a-99547b1fd72d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:53.918250 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f9afa34a-ffba-4ba5-b03a-99547b1fd72d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:53.918708 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f9afa34a-ffba-4ba5-b03a-99547b1fd72d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:53.919549 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f9afa34a-ffba-4ba5-b03a-99547b1fd72d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:53.919992 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-f9afa34a-ffba-4ba5-b03a-99547b1fd72d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:54.052171 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-f9afa34a-ffba-4ba5-b03a-99547b1fd72d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:54.059502 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-f9afa34a-ffba-4ba5-b03a-99547b1fd72d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1567 microversion: 2.1 time: 0.152741 Jul 19 11:11:54.060066 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 569/1132] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:53 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1567 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:54.162549 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-b111cd08-0e53-4095-9e58-d37077e03c3a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:54.163083 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-b111cd08-0e53-4095-9e58-d37077e03c3a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:54.169095 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b111cd08-0e53-4095-9e58-d37077e03c3a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:54.169546 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b111cd08-0e53-4095-9e58-d37077e03c3a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:54.169670 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-dec6399f-52b8-40f5-bb9a-446de58b616b tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:54.170307 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b111cd08-0e53-4095-9e58-d37077e03c3a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:54.170565 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-dec6399f-52b8-40f5-bb9a-446de58b616b tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:54.170740 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b111cd08-0e53-4095-9e58-d37077e03c3a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:54.176551 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dec6399f-52b8-40f5-bb9a-446de58b616b tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:54.177025 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dec6399f-52b8-40f5-bb9a-446de58b616b tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:54.177782 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dec6399f-52b8-40f5-bb9a-446de58b616b tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:54.178242 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dec6399f-52b8-40f5-bb9a-446de58b616b tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:54.206793 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-dec6399f-52b8-40f5-bb9a-446de58b616b tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041003 Jul 19 11:11:54.207358 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 570/1133] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:11:54 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:11:54.553784 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b111cd08-0e53-4095-9e58-d37077e03c3a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:54.561368 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-b111cd08-0e53-4095-9e58-d37077e03c3a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.402712 Jul 19 11:11:54.561940 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 564/1134] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:54 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:54.926786 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-8aed3795-ade1-4e2f-aa11-b5665665d29b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:54.927394 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-8aed3795-ade1-4e2f-aa11-b5665665d29b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:54.933894 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8aed3795-ade1-4e2f-aa11-b5665665d29b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:54.934341 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8aed3795-ade1-4e2f-aa11-b5665665d29b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:54.935102 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8aed3795-ade1-4e2f-aa11-b5665665d29b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:54.935547 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8aed3795-ade1-4e2f-aa11-b5665665d29b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:55.066848 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-8aed3795-ade1-4e2f-aa11-b5665665d29b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:55.075133 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-8aed3795-ade1-4e2f-aa11-b5665665d29b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:55.076245 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-b167fe58-d74b-4fad-931e-8688e36d3ed8 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:55.076646 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-8aed3795-ade1-4e2f-aa11-b5665665d29b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:55.076938 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-b167fe58-d74b-4fad-931e-8688e36d3ed8 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:55.077856 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-8aed3795-ade1-4e2f-aa11-b5665665d29b tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.155511 Jul 19 11:11:55.084743 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 571/1135] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:54 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:55.084900 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b167fe58-d74b-4fad-931e-8688e36d3ed8 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:55.084900 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b167fe58-d74b-4fad-931e-8688e36d3ed8 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:55.084900 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b167fe58-d74b-4fad-931e-8688e36d3ed8 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:55.085209 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b167fe58-d74b-4fad-931e-8688e36d3ed8 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:55.217674 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-38cf3332-e97c-4548-8ccb-ba5414939e3d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:55.217939 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b167fe58-d74b-4fad-931e-8688e36d3ed8 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:55.218703 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-38cf3332-e97c-4548-8ccb-ba5414939e3d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:55.220387 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=100 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:11:55.225618 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-38cf3332-e97c-4548-8ccb-ba5414939e3d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:55.226065 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-38cf3332-e97c-4548-8ccb-ba5414939e3d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:55.226834 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-38cf3332-e97c-4548-8ccb-ba5414939e3d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:55.227267 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-38cf3332-e97c-4548-8ccb-ba5414939e3d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:55.228790 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-b167fe58-d74b-4fad-931e-8688e36d3ed8 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1567 microversion: 2.1 time: 0.156223 Jul 19 11:11:55.231059 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 565/1136] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:55 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1567 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:11:55.258543 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-38cf3332-e97c-4548-8ccb-ba5414939e3d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044763 Jul 19 11:11:55.259311 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 572/1137] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:11:55 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:11:55.578517 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-0cb79c00-d47f-4054-ae47-2cfcf9abec02 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:55.579062 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-0cb79c00-d47f-4054-ae47-2cfcf9abec02 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:55.585777 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0cb79c00-d47f-4054-ae47-2cfcf9abec02 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:55.586229 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0cb79c00-d47f-4054-ae47-2cfcf9abec02 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:55.586993 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0cb79c00-d47f-4054-ae47-2cfcf9abec02 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:55.587435 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0cb79c00-d47f-4054-ae47-2cfcf9abec02 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:55.717682 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-0cb79c00-d47f-4054-ae47-2cfcf9abec02 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:55.725827 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-0cb79c00-d47f-4054-ae47-2cfcf9abec02 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.151213 Jul 19 11:11:55.726396 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 566/1138] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:55 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:56.095646 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-e203b83f-9891-4e96-93f2-9ceaa2a9537f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:56.096717 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-e203b83f-9891-4e96-93f2-9ceaa2a9537f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:56.103197 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e203b83f-9891-4e96-93f2-9ceaa2a9537f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:56.103640 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e203b83f-9891-4e96-93f2-9ceaa2a9537f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:56.104397 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e203b83f-9891-4e96-93f2-9ceaa2a9537f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:56.104853 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e203b83f-9891-4e96-93f2-9ceaa2a9537f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:56.232940 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-e203b83f-9891-4e96-93f2-9ceaa2a9537f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:56.240707 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-e203b83f-9891-4e96-93f2-9ceaa2a9537f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:56.242214 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-e203b83f-9891-4e96-93f2-9ceaa2a9537f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:56.243350 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-e203b83f-9891-4e96-93f2-9ceaa2a9537f tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.151654 Jul 19 11:11:56.244084 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 573/1139] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:56 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:56.246398 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-586d5abe-8a1f-4ed2-8a90-d1eb9931454d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:56.247002 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-586d5abe-8a1f-4ed2-8a90-d1eb9931454d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:56.253705 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-586d5abe-8a1f-4ed2-8a90-d1eb9931454d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:56.254156 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-586d5abe-8a1f-4ed2-8a90-d1eb9931454d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:56.254914 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-586d5abe-8a1f-4ed2-8a90-d1eb9931454d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:56.255351 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-586d5abe-8a1f-4ed2-8a90-d1eb9931454d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:56.270506 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-d0066f58-9952-4c8d-9ea6-23fa10acf5f5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:56.271448 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-d0066f58-9952-4c8d-9ea6-23fa10acf5f5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:56.277302 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d0066f58-9952-4c8d-9ea6-23fa10acf5f5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:56.277746 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d0066f58-9952-4c8d-9ea6-23fa10acf5f5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:56.278500 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d0066f58-9952-4c8d-9ea6-23fa10acf5f5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:56.278929 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d0066f58-9952-4c8d-9ea6-23fa10acf5f5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:56.308400 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-d0066f58-9952-4c8d-9ea6-23fa10acf5f5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042252 Jul 19 11:11:56.308994 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 574/1140] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:11:56 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:11:56.384005 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-586d5abe-8a1f-4ed2-8a90-d1eb9931454d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:56.391550 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-586d5abe-8a1f-4ed2-8a90-d1eb9931454d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1567 microversion: 2.1 time: 0.148967 Jul 19 11:11:56.392104 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 567/1141] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:56 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1567 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:11:56.743013 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-ed4c16de-9ff7-437d-afe9-3e45a5374806 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:56.743609 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-ed4c16de-9ff7-437d-afe9-3e45a5374806 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:56.749508 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ed4c16de-9ff7-437d-afe9-3e45a5374806 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:56.749951 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ed4c16de-9ff7-437d-afe9-3e45a5374806 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:56.750952 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ed4c16de-9ff7-437d-afe9-3e45a5374806 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:56.751381 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ed4c16de-9ff7-437d-afe9-3e45a5374806 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:56.894898 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-ed4c16de-9ff7-437d-afe9-3e45a5374806 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:56.902237 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-ed4c16de-9ff7-437d-afe9-3e45a5374806 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.163759 Jul 19 11:11:56.902798 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 575/1142] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:56 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:57.261394 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-dfbdd7c3-fdad-4454-b14d-8c9b2b91217a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:57.262004 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-dfbdd7c3-fdad-4454-b14d-8c9b2b91217a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:57.268496 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dfbdd7c3-fdad-4454-b14d-8c9b2b91217a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:57.268978 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dfbdd7c3-fdad-4454-b14d-8c9b2b91217a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:57.269745 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dfbdd7c3-fdad-4454-b14d-8c9b2b91217a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:57.270177 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-dfbdd7c3-fdad-4454-b14d-8c9b2b91217a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:57.319968 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-760e631c-5a87-4767-b3ea-3971c2298fb8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:57.321028 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-760e631c-5a87-4767-b3ea-3971c2298fb8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:57.327580 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-760e631c-5a87-4767-b3ea-3971c2298fb8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:57.328027 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-760e631c-5a87-4767-b3ea-3971c2298fb8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:57.328810 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-760e631c-5a87-4767-b3ea-3971c2298fb8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:57.329241 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-760e631c-5a87-4767-b3ea-3971c2298fb8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:57.358927 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-760e631c-5a87-4767-b3ea-3971c2298fb8 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042892 Jul 19 11:11:57.359508 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 576/1143] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:11:57 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:11:57.398418 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-dfbdd7c3-fdad-4454-b14d-8c9b2b91217a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:57.406644 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-dfbdd7c3-fdad-4454-b14d-8c9b2b91217a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:57.408154 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-dfbdd7c3-fdad-4454-b14d-8c9b2b91217a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:57.409583 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-cb49fa85-e590-42ee-9799-4c62c1c77b00 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:57.409857 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-dfbdd7c3-fdad-4454-b14d-8c9b2b91217a tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.152390 Jul 19 11:11:57.410226 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-cb49fa85-e590-42ee-9799-4c62c1c77b00 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:57.410416 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 568/1144] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:57 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:57.416243 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cb49fa85-e590-42ee-9799-4c62c1c77b00 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:57.416712 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cb49fa85-e590-42ee-9799-4c62c1c77b00 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:57.417519 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cb49fa85-e590-42ee-9799-4c62c1c77b00 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:57.417953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cb49fa85-e590-42ee-9799-4c62c1c77b00 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:57.544740 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-cb49fa85-e590-42ee-9799-4c62c1c77b00 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:57.553366 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-cb49fa85-e590-42ee-9799-4c62c1c77b00 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1567 microversion: 2.1 time: 0.148136 Jul 19 11:11:57.553933 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 577/1145] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:57 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1567 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:11:57.918879 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-5eba81ed-888f-4944-977d-cece61aa5541 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:57.919484 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-5eba81ed-888f-4944-977d-cece61aa5541 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:57.925577 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5eba81ed-888f-4944-977d-cece61aa5541 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:57.926026 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5eba81ed-888f-4944-977d-cece61aa5541 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:57.926788 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5eba81ed-888f-4944-977d-cece61aa5541 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:57.927225 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5eba81ed-888f-4944-977d-cece61aa5541 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:58.060124 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-5eba81ed-888f-4944-977d-cece61aa5541 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:58.068120 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-5eba81ed-888f-4944-977d-cece61aa5541 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.153216 Jul 19 11:11:58.068684 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 569/1146] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:57 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:58.371170 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-7524cfbb-bccd-4b90-afdf-1294c9c2ead9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:58.372131 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-7524cfbb-bccd-4b90-afdf-1294c9c2ead9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:58.379325 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7524cfbb-bccd-4b90-afdf-1294c9c2ead9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:58.379765 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7524cfbb-bccd-4b90-afdf-1294c9c2ead9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:58.380516 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7524cfbb-bccd-4b90-afdf-1294c9c2ead9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:58.380975 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7524cfbb-bccd-4b90-afdf-1294c9c2ead9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:58.410642 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-7524cfbb-bccd-4b90-afdf-1294c9c2ead9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043474 Jul 19 11:11:58.411202 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 578/1147] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:11:58 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:11:58.428588 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-4783afca-cde3-4e52-9c7a-9f11b1522eff tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:58.429219 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-4783afca-cde3-4e52-9c7a-9f11b1522eff tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:58.435704 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4783afca-cde3-4e52-9c7a-9f11b1522eff tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:58.436151 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4783afca-cde3-4e52-9c7a-9f11b1522eff tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:58.436934 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4783afca-cde3-4e52-9c7a-9f11b1522eff tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:58.437367 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4783afca-cde3-4e52-9c7a-9f11b1522eff tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:58.569367 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-4783afca-cde3-4e52-9c7a-9f11b1522eff tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:58.570745 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-0cae0882-83df-447d-b9da-3df0e3ce75f2 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:58.578068 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-4783afca-cde3-4e52-9c7a-9f11b1522eff tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:58.579005 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-0cae0882-83df-447d-b9da-3df0e3ce75f2 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:58.579568 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-4783afca-cde3-4e52-9c7a-9f11b1522eff tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:58.580735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-4783afca-cde3-4e52-9c7a-9f11b1522eff tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.157162 Jul 19 11:11:58.581293 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 570/1148] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:58 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:11:58.585190 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0cae0882-83df-447d-b9da-3df0e3ce75f2 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:58.585631 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0cae0882-83df-447d-b9da-3df0e3ce75f2 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:58.586391 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0cae0882-83df-447d-b9da-3df0e3ce75f2 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:58.586823 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0cae0882-83df-447d-b9da-3df0e3ce75f2 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:58.773619 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-0cae0882-83df-447d-b9da-3df0e3ce75f2 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:58.781977 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-0cae0882-83df-447d-b9da-3df0e3ce75f2 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1567 microversion: 2.1 time: 0.215008 Jul 19 11:11:58.782547 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 579/1149] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:58 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1567 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:11:59.086308 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-5066e2d0-fa63-4bdd-a7b0-191782dba98a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:59.086913 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-5066e2d0-fa63-4bdd-a7b0-191782dba98a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:59.092943 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5066e2d0-fa63-4bdd-a7b0-191782dba98a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:59.093392 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5066e2d0-fa63-4bdd-a7b0-191782dba98a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:59.094147 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5066e2d0-fa63-4bdd-a7b0-191782dba98a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:59.094647 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5066e2d0-fa63-4bdd-a7b0-191782dba98a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:59.230923 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-5066e2d0-fa63-4bdd-a7b0-191782dba98a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:59.239286 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-5066e2d0-fa63-4bdd-a7b0-191782dba98a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1267 microversion: 2.1 time: 0.157643 Jul 19 11:11:59.239852 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 571/1150] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:59 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1267 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:11:59.421721 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-47323dd4-aa83-4966-80d7-7fc7c3757744 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:59.422636 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-47323dd4-aa83-4966-80d7-7fc7c3757744 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:59.429706 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-47323dd4-aa83-4966-80d7-7fc7c3757744 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:59.430091 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-47323dd4-aa83-4966-80d7-7fc7c3757744 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:59.430837 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-47323dd4-aa83-4966-80d7-7fc7c3757744 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:59.431266 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-47323dd4-aa83-4966-80d7-7fc7c3757744 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:59.459705 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-47323dd4-aa83-4966-80d7-7fc7c3757744 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041942 Jul 19 11:11:59.460239 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 580/1151] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:11:59 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:11:59.599312 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-d4e6e30f-8399-4746-b6c6-d25725da0373 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:59.599918 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-d4e6e30f-8399-4746-b6c6-d25725da0373 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:59.606362 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d4e6e30f-8399-4746-b6c6-d25725da0373 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:59.606811 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d4e6e30f-8399-4746-b6c6-d25725da0373 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:59.607571 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d4e6e30f-8399-4746-b6c6-d25725da0373 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:59.608005 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d4e6e30f-8399-4746-b6c6-d25725da0373 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:59.769645 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-d4e6e30f-8399-4746-b6c6-d25725da0373 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:59.778679 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-d4e6e30f-8399-4746-b6c6-d25725da0373 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:59.780280 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-d4e6e30f-8399-4746-b6c6-d25725da0373 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:59.781750 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-d4e6e30f-8399-4746-b6c6-d25725da0373 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.186826 Jul 19 11:11:59.793799 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 572/1152] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:11:59 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:11:59.798850 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-1a93d973-3b5c-4dbe-b9ab-77caed26cd1d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:11:59.799824 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-1a93d973-3b5c-4dbe-b9ab-77caed26cd1d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:11:59.806558 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1a93d973-3b5c-4dbe-b9ab-77caed26cd1d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:59.807096 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1a93d973-3b5c-4dbe-b9ab-77caed26cd1d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:59.808050 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1a93d973-3b5c-4dbe-b9ab-77caed26cd1d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:11:59.808492 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1a93d973-3b5c-4dbe-b9ab-77caed26cd1d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:11:59.813179 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=100 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:11:59.941598 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-1a93d973-3b5c-4dbe-b9ab-77caed26cd1d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:11:59.949097 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-1a93d973-3b5c-4dbe-b9ab-77caed26cd1d tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1567 microversion: 2.1 time: 0.154041 Jul 19 11:11:59.949664 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 581/1153] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:11:59 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1567 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:12:00.195794 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-61c60b7e-db6c-40ec-9991-6404a86e24a7 req-e4543c0f-b31a-4246-a6d8-f6625fe5ecf9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5b72a14b-bd18-4dfa-a68f-12a5424f471c", "name": "network-vif-deleted", "tag": "da843419-4401-4a26-82c2-7930b51aa1b8"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:12:00.203832 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-61c60b7e-db6c-40ec-9991-6404a86e24a7 req-e4543c0f-b31a-4246-a6d8-f6625fe5ecf9 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:00.204282 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-61c60b7e-db6c-40ec-9991-6404a86e24a7 req-e4543c0f-b31a-4246-a6d8-f6625fe5ecf9 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:00.215088 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-61c60b7e-db6c-40ec-9991-6404a86e24a7 req-e4543c0f-b31a-4246-a6d8-f6625fe5ecf9 service nova] Creating event network-vif-deleted:da843419-4401-4a26-82c2-7930b51aa1b8 for instance 5b72a14b-bd18-4dfa-a68f-12a5424f471c on hv-cinder-maste Jul 19 11:12:00.220481 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-61c60b7e-db6c-40ec-9991-6404a86e24a7 req-e4543c0f-b31a-4246-a6d8-f6625fe5ecf9 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.028267 Jul 19 11:12:00.221136 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 573/1154] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:12:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:12:00.256371 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-b9abfa70-64ae-459c-8cdd-e63783204dfb tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:00.257012 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-b9abfa70-64ae-459c-8cdd-e63783204dfb tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:00.262967 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b9abfa70-64ae-459c-8cdd-e63783204dfb tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:00.263409 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b9abfa70-64ae-459c-8cdd-e63783204dfb tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:00.264164 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b9abfa70-64ae-459c-8cdd-e63783204dfb tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:00.264591 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b9abfa70-64ae-459c-8cdd-e63783204dfb tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:00.393459 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-b9abfa70-64ae-459c-8cdd-e63783204dfb tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:12:00.401133 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-b9abfa70-64ae-459c-8cdd-e63783204dfb tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1455 microversion: 2.1 time: 0.149014 Jul 19 11:12:00.401711 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 582/1155] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:12:00 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1455 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:12:00.418166 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-e0bc40f5-1414-4d2f-9eac-0ba82df22f0c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:00.418860 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-e0bc40f5-1414-4d2f-9eac-0ba82df22f0c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:00.425666 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e0bc40f5-1414-4d2f-9eac-0ba82df22f0c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:00.426116 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e0bc40f5-1414-4d2f-9eac-0ba82df22f0c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:00.426866 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e0bc40f5-1414-4d2f-9eac-0ba82df22f0c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:00.427301 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-e0bc40f5-1414-4d2f-9eac-0ba82df22f0c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:00.471816 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-d8b655dc-85f2-4c32-987d-f5b83e31f94b tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:00.473396 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-d8b655dc-85f2-4c32-987d-f5b83e31f94b tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:00.479832 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d8b655dc-85f2-4c32-987d-f5b83e31f94b tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:00.480276 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d8b655dc-85f2-4c32-987d-f5b83e31f94b tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:00.481083 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d8b655dc-85f2-4c32-987d-f5b83e31f94b tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:00.481513 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d8b655dc-85f2-4c32-987d-f5b83e31f94b tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:00.514810 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-d8b655dc-85f2-4c32-987d-f5b83e31f94b tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.047392 Jul 19 11:12:00.515615 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 583/1156] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:12:00 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67/os-volume_attachments => generated 25 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Jul 19 11:12:00.567645 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-e0bc40f5-1414-4d2f-9eac-0ba82df22f0c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:12:00.574327 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-33eb15ed-8592-47cd-9936-7ccea31f3e7e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:00.575062 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-33eb15ed-8592-47cd-9936-7ccea31f3e7e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:00.575866 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-e0bc40f5-1414-4d2f-9eac-0ba82df22f0c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1455 microversion: 2.1 time: 0.161776 Jul 19 11:12:00.576540 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 574/1157] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:12:00 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1455 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:12:00.582538 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-33eb15ed-8592-47cd-9936-7ccea31f3e7e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:00.582984 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-33eb15ed-8592-47cd-9936-7ccea31f3e7e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:00.583758 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-33eb15ed-8592-47cd-9936-7ccea31f3e7e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:00.584188 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-33eb15ed-8592-47cd-9936-7ccea31f3e7e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:00.610815 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-33eb15ed-8592-47cd-9936-7ccea31f3e7e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Going to try to terminate instance {{(pid=117848) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 19 11:12:00.696039 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-33eb15ed-8592-47cd-9936-7ccea31f3e7e tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "DELETE /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67" status: 204 len: 0 microversion: 2.1 time: 0.125314 Jul 19 11:12:00.696807 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 584/1158] 10.222.0.33 () {58 vars in 1238 bytes} [Tue Jul 19 11:12:00 2022] DELETE /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 19 11:12:00.706147 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-d7e780a1-444d-4f7f-b37e-c4901b997502 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:00.707329 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-d7e780a1-444d-4f7f-b37e-c4901b997502 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:00.714513 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d7e780a1-444d-4f7f-b37e-c4901b997502 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:00.714973 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d7e780a1-444d-4f7f-b37e-c4901b997502 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:00.715899 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d7e780a1-444d-4f7f-b37e-c4901b997502 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:00.716340 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-d7e780a1-444d-4f7f-b37e-c4901b997502 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:00.801127 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-e8cd1686-8756-4e02-99d4-b490adb42f5c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:00.802188 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-e8cd1686-8756-4e02-99d4-b490adb42f5c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:00.809319 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e8cd1686-8756-4e02-99d4-b490adb42f5c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:00.810051 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e8cd1686-8756-4e02-99d4-b490adb42f5c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:00.811052 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e8cd1686-8756-4e02-99d4-b490adb42f5c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:00.811675 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e8cd1686-8756-4e02-99d4-b490adb42f5c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:00.921945 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-d7e780a1-444d-4f7f-b37e-c4901b997502 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:12:00.931128 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-d7e780a1-444d-4f7f-b37e-c4901b997502 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67" status: 200 len: 1681 microversion: 2.1 time: 0.229331 Jul 19 11:12:00.942922 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 575/1159] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:12:00 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67 => generated 1681 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:12:00.969653 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-6ee48868-502e-4c81-849d-caf4061b6b64 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:00.970515 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-6ee48868-502e-4c81-849d-caf4061b6b64 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:00.978748 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6ee48868-502e-4c81-849d-caf4061b6b64 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:00.979677 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6ee48868-502e-4c81-849d-caf4061b6b64 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:00.980961 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6ee48868-502e-4c81-849d-caf4061b6b64 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:00.981646 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6ee48868-502e-4c81-849d-caf4061b6b64 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:01.058115 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-6ee48868-502e-4c81-849d-caf4061b6b64 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:12:01.066275 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-6ee48868-502e-4c81-849d-caf4061b6b64 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1218 microversion: 2.1 time: 0.103007 Jul 19 11:12:01.066932 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 576/1160] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:12:00 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1218 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:12:01.282307 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-e8cd1686-8756-4e02-99d4-b490adb42f5c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:12:01.292147 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-e8cd1686-8756-4e02-99d4-b490adb42f5c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:12:01.293941 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-e8cd1686-8756-4e02-99d4-b490adb42f5c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:12:01.295378 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-e8cd1686-8756-4e02-99d4-b490adb42f5c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.498741 Jul 19 11:12:01.296370 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 585/1161] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:12:00 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 500 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:12:01.948615 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-b0d35ec3-9b03-425e-8d3c-d6047acca7b5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:01.949246 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-b0d35ec3-9b03-425e-8d3c-d6047acca7b5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:01.956337 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b0d35ec3-9b03-425e-8d3c-d6047acca7b5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:01.956819 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b0d35ec3-9b03-425e-8d3c-d6047acca7b5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:01.957576 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b0d35ec3-9b03-425e-8d3c-d6047acca7b5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:01.958014 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b0d35ec3-9b03-425e-8d3c-d6047acca7b5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:02.084051 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-352518ad-8a21-4c3c-b275-b36c6917ba1e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:02.084654 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-352518ad-8a21-4c3c-b275-b36c6917ba1e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:02.103791 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b0d35ec3-9b03-425e-8d3c-d6047acca7b5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:12:02.106625 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-352518ad-8a21-4c3c-b275-b36c6917ba1e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:02.107046 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-352518ad-8a21-4c3c-b275-b36c6917ba1e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:02.107812 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-352518ad-8a21-4c3c-b275-b36c6917ba1e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:02.108248 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-352518ad-8a21-4c3c-b275-b36c6917ba1e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:02.114158 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-b0d35ec3-9b03-425e-8d3c-d6047acca7b5 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67" status: 200 len: 1681 microversion: 2.1 time: 0.169329 Jul 19 11:12:02.121279 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 577/1162] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:12:01 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67 => generated 1681 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:12:02.191993 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-352518ad-8a21-4c3c-b275-b36c6917ba1e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:12:02.200748 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-352518ad-8a21-4c3c-b275-b36c6917ba1e tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1218 microversion: 2.1 time: 0.120468 Jul 19 11:12:02.201422 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 586/1163] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:12:02 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1218 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:12:02.315781 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-7de63e4f-29e9-44d6-bf26-9e906ae53d2c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:02.317217 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-7de63e4f-29e9-44d6-bf26-9e906ae53d2c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:02.324128 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7de63e4f-29e9-44d6-bf26-9e906ae53d2c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:02.324712 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7de63e4f-29e9-44d6-bf26-9e906ae53d2c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:02.325647 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7de63e4f-29e9-44d6-bf26-9e906ae53d2c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:02.326194 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7de63e4f-29e9-44d6-bf26-9e906ae53d2c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:02.482526 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-7de63e4f-29e9-44d6-bf26-9e906ae53d2c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:12:02.493514 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-7de63e4f-29e9-44d6-bf26-9e906ae53d2c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:12:02.495182 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-7de63e4f-29e9-44d6-bf26-9e906ae53d2c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:12:02.496600 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-7de63e4f-29e9-44d6-bf26-9e906ae53d2c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1730 microversion: 2.20 time: 0.185615 Jul 19 11:12:02.497287 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 578/1164] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:12:02 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1730 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:12:02.849170 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-59340aac-80e0-47e6-868b-3ac6e5f75df6 req-8bbae587-3640-4fa8-8f1a-c2ed650edffb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2", "tag": "29b83b99-dced-4119-b3f1-cb6ec3d0c385"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:12:02.857242 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-59340aac-80e0-47e6-868b-3ac6e5f75df6 req-8bbae587-3640-4fa8-8f1a-c2ed650edffb service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:02.857782 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-59340aac-80e0-47e6-868b-3ac6e5f75df6 req-8bbae587-3640-4fa8-8f1a-c2ed650edffb service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:02.868886 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-59340aac-80e0-47e6-868b-3ac6e5f75df6 req-8bbae587-3640-4fa8-8f1a-c2ed650edffb service nova] Creating event network-changed:29b83b99-dced-4119-b3f1-cb6ec3d0c385 for instance 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 on hv-cinder-maste Jul 19 11:12:02.873818 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-59340aac-80e0-47e6-868b-3ac6e5f75df6 req-8bbae587-3640-4fa8-8f1a-c2ed650edffb service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.028367 Jul 19 11:12:02.874422 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 587/1165] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:12:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:12:03.131693 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-47b7d9f9-560f-4a0c-ade7-30815b935153 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:03.132217 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-47b7d9f9-560f-4a0c-ade7-30815b935153 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:03.138903 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-47b7d9f9-560f-4a0c-ade7-30815b935153 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:03.139460 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-47b7d9f9-560f-4a0c-ade7-30815b935153 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:03.140365 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-47b7d9f9-560f-4a0c-ade7-30815b935153 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:03.140912 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-47b7d9f9-560f-4a0c-ade7-30815b935153 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:03.222489 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-ddcc7f68-169a-4d73-bb99-aaae1989b4a0 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:03.223201 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-ddcc7f68-169a-4d73-bb99-aaae1989b4a0 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:03.230747 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ddcc7f68-169a-4d73-bb99-aaae1989b4a0 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:03.231201 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ddcc7f68-169a-4d73-bb99-aaae1989b4a0 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:03.231975 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ddcc7f68-169a-4d73-bb99-aaae1989b4a0 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:03.232414 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-ddcc7f68-169a-4d73-bb99-aaae1989b4a0 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:03.290459 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-47b7d9f9-560f-4a0c-ade7-30815b935153 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:12:03.298994 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-47b7d9f9-560f-4a0c-ade7-30815b935153 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67" status: 200 len: 1681 microversion: 2.1 time: 0.171124 Jul 19 11:12:03.299831 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 579/1166] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:12:03 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67 => generated 1681 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:12:03.322818 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-ddcc7f68-169a-4d73-bb99-aaae1989b4a0 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ece5061796245a1b2a6bef1d9d6323a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '375de853371a422f883980296a46ad43', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:12:03.334558 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-ddcc7f68-169a-4d73-bb99-aaae1989b4a0 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 200 len: 1381 microversion: 2.1 time: 0.119370 Jul 19 11:12:03.344809 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 588/1167] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:12:03 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 1381 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:12:03.516330 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-7cf1e8a1-63f2-4f0c-9d9e-a53c194eb8f9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:03.517256 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-7cf1e8a1-63f2-4f0c-9d9e-a53c194eb8f9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:03.539647 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7cf1e8a1-63f2-4f0c-9d9e-a53c194eb8f9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:03.540474 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7cf1e8a1-63f2-4f0c-9d9e-a53c194eb8f9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:03.541380 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7cf1e8a1-63f2-4f0c-9d9e-a53c194eb8f9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:03.542658 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7cf1e8a1-63f2-4f0c-9d9e-a53c194eb8f9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:03.708063 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-7cf1e8a1-63f2-4f0c-9d9e-a53c194eb8f9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:12:03.720506 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-7cf1e8a1-63f2-4f0c-9d9e-a53c194eb8f9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:12:03.722760 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-7cf1e8a1-63f2-4f0c-9d9e-a53c194eb8f9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:12:03.724330 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-7cf1e8a1-63f2-4f0c-9d9e-a53c194eb8f9 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1717 microversion: 2.20 time: 0.212618 Jul 19 11:12:03.736723 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 580/1168] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:12:03 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1717 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:12:04.317081 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-15912460-82cf-4139-ae9b-dee37e79efec tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:04.317694 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-15912460-82cf-4139-ae9b-dee37e79efec tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:04.321229 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:12:04.326921 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-15912460-82cf-4139-ae9b-dee37e79efec tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:04.327495 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-15912460-82cf-4139-ae9b-dee37e79efec tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:04.328394 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-15912460-82cf-4139-ae9b-dee37e79efec tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:04.328925 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-15912460-82cf-4139-ae9b-dee37e79efec tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:04.352484 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-5ea6f464-bac8-47ba-809d-0d5b9c05f757 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:04.354026 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-5ea6f464-bac8-47ba-809d-0d5b9c05f757 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] [instance: 5b72a14b-bd18-4dfa-a68f-12a5424f471c] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:04.360795 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5ea6f464-bac8-47ba-809d-0d5b9c05f757 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:04.364351 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5ea6f464-bac8-47ba-809d-0d5b9c05f757 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:04.365165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5ea6f464-bac8-47ba-809d-0d5b9c05f757 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:04.365603 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5ea6f464-bac8-47ba-809d-0d5b9c05f757 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:04.392764 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.wsgi [None req-5ea6f464-bac8-47ba-809d-0d5b9c05f757 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] HTTP exception thrown: Instance 5b72a14b-bd18-4dfa-a68f-12a5424f471c could not be found. Jul 19 11:12:04.395219 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-5ea6f464-bac8-47ba-809d-0d5b9c05f757 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Request method failure captured: Jul 19 11:12:04.395219 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: request: GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c HTTP/1.1 Jul 19 11:12:04.395219 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept: application/json Jul 19 11:12:04.395219 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept-Encoding: identity Jul 19 11:12:04.395219 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Connection: close Jul 19 11:12:04.395219 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Length: 0 Jul 19 11:12:04.395219 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Type: application/json Jul 19 11:12:04.395219 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Host: 10.222.0.33 Jul 19 11:12:04.395219 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Openstack-System-Scope: None Jul 19 11:12:04.395219 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: User-Agent: python-urllib3/1.26.9 Jul 19 11:12:04.395219 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Auth-Token: gAAAAABi1pAb9bvTWoTGaw8ncRe9i28u-nNnE6jdN7TZwu4k7tN3LArtrIcScGTdcgxDx-5j3czid7rX3DWdkaTWM35TlzbbpeCW1wszgu42yaHk07C2X1A8n7WwKq6qWmx7sm2pmacvB0OeqjTGPYY-2TBiejckNe4kB5vHi8GjqV552P_ooZs Jul 19 11:12:04.395219 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Id: None Jul 19 11:12:04.395219 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Name: None Jul 19 11:12:04.395219 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Identity-Status: Confirmed Jul 19 11:12:04.395219 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Is-Admin-Project: True Jul 19 11:12:04.395219 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Id: default Jul 19 11:12:04.395219 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Name: Default Jul 19 11:12:04.395219 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Id: 375de853371a422f883980296a46ad43 Jul 19 11:12:04.395219 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Name: tempest-TestShelveInstance-804313476 Jul 19 11:12:04.395219 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Role: member,reader Jul 19 11:12:04.395219 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Roles: member,reader Jul 19 11:12:04.398045 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/375de853371a422f883980296a46ad43"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/375de853371a422f883980296a46ad43"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/375de853371a422f883980296a46ad43"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_375de853371a422f883980296a46ad43"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:12:04.398045 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant: tempest-TestShelveInstance-804313476 Jul 19 11:12:04.398045 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Id: 375de853371a422f883980296a46ad43 Jul 19 11:12:04.398045 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Name: tempest-TestShelveInstance-804313476 Jul 19 11:12:04.398045 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User: tempest-TestShelveInstance-804313476-project Jul 19 11:12:04.398045 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Id: default Jul 19 11:12:04.398045 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Name: Default Jul 19 11:12:04.398045 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Id: 3ece5061796245a1b2a6bef1d9d6323a Jul 19 11:12:04.398045 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Name: tempest-TestShelveInstance-804313476-project Jul 19 11:12:04.398045 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: method: > Jul 19 11:12:04.398045 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: exception: Instance 5b72a14b-bd18-4dfa-a68f-12a5424f471c could not be found. Jul 19 11:12:04.398045 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: action_args: {'id': '5b72a14b-bd18-4dfa-a68f-12a5424f471c'} Jul 19 11:12:04.399735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:12:04.399735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:04.399735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:12:04.399735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:12:04.399735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:12:04.399735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:12:04.399735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:12:04.399735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:12:04.399735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:12:04.399735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:12:04.399735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5b72a14b-bd18-4dfa-a68f-12a5424f471c could not be found. Jul 19 11:12:04.399735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:04.399735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:04.399735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:04.399735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:04.399735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:04.399735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:04.399735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:12:04.399735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:12:04.399735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:12:04.402278 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:12:04.402278 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 19 11:12:04.402278 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 19 11:12:04.402278 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:12:04.402278 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:12:04.402278 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:12:04.402278 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:12:04.402278 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5b72a14b-bd18-4dfa-a68f-12a5424f471c could not be found. Jul 19 11:12:04.402278 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:04.402278 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:04.402278 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:04.402278 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:04.402278 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:04.402278 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:04.402278 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:12:04.402278 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:12:04.402278 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5b72a14b-bd18-4dfa-a68f-12a5424f471c could not be found. Jul 19 11:12:04.402278 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:04.407473 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-5ea6f464-bac8-47ba-809d-0d5b9c05f757 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Returning 404 to user: Instance 5b72a14b-bd18-4dfa-a68f-12a5424f471c could not be found. {{(pid=117849) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:12:04.407473 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-5ea6f464-bac8-47ba-809d-0d5b9c05f757 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c" status: 404 len: 111 microversion: 2.1 time: 0.047931 Jul 19 11:12:04.407473 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 581/1169] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:12:04 2022] GET /compute/v2.1/servers/5b72a14b-bd18-4dfa-a68f-12a5424f471c => generated 111 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 19 11:12:04.443881 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-15912460-82cf-4139-ae9b-dee37e79efec tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8f2a78f3a3e4b8e8937e3bc39524690', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef534f1d06ea415d98857ec7f9e5fb97', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:12:04.453527 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-15912460-82cf-4139-ae9b-dee37e79efec tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67" status: 200 len: 1613 microversion: 2.1 time: 0.141063 Jul 19 11:12:04.454749 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 589/1170] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:12:04 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67 => generated 1613 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:12:04.624896 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-279f7ed4-771a-482d-b3d8-4bd93f224aa8 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] Calling method '.version_select at 0x7f04aa48c0d0>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:04.644469 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-279f7ed4-771a-482d-b3d8-4bd93f224aa8 tempest-TestShelveInstance-804313476 tempest-TestShelveInstance-804313476-project] 10.222.0.33 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-261123105" status: 202 len: 0 microversion: 2.1 time: 0.023323 Jul 19 11:12:04.645176 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 582/1171] 10.222.0.33 () {58 vars in 1250 bytes} [Tue Jul 19 11:12:04 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-261123105 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 19 11:12:04.897149 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-1bfb5ca7-f932-401c-9b61-bb11f440ffee req-a173b46d-783a-4a27-bcc0-8a1b2a273901 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e9cecc8a-4c06-454a-91b6-0b735a222e67", "name": "network-vif-deleted", "tag": "8bc77a5e-bfb6-4334-8a9a-3ebaf41dd8e5"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:12:04.914601 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-1bfb5ca7-f932-401c-9b61-bb11f440ffee req-a173b46d-783a-4a27-bcc0-8a1b2a273901 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:04.915280 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-1bfb5ca7-f932-401c-9b61-bb11f440ffee req-a173b46d-783a-4a27-bcc0-8a1b2a273901 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:04.931548 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-1bfb5ca7-f932-401c-9b61-bb11f440ffee req-a173b46d-783a-4a27-bcc0-8a1b2a273901 service nova] Creating event network-vif-deleted:8bc77a5e-bfb6-4334-8a9a-3ebaf41dd8e5 for instance e9cecc8a-4c06-454a-91b6-0b735a222e67 on hv-cinder-maste Jul 19 11:12:04.938344 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-1bfb5ca7-f932-401c-9b61-bb11f440ffee req-a173b46d-783a-4a27-bcc0-8a1b2a273901 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044772 Jul 19 11:12:04.938962 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 590/1172] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:12:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:12:05.473158 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-f61b6c58-4d70-428f-82c0-fab4916e5336 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:05.474002 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-f61b6c58-4d70-428f-82c0-fab4916e5336 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:05.482945 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f61b6c58-4d70-428f-82c0-fab4916e5336 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:05.483505 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f61b6c58-4d70-428f-82c0-fab4916e5336 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:05.484895 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f61b6c58-4d70-428f-82c0-fab4916e5336 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:05.485436 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f61b6c58-4d70-428f-82c0-fab4916e5336 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:05.502907 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.wsgi [None req-f61b6c58-4d70-428f-82c0-fab4916e5336 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] HTTP exception thrown: Instance e9cecc8a-4c06-454a-91b6-0b735a222e67 could not be found. Jul 19 11:12:05.504592 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-f61b6c58-4d70-428f-82c0-fab4916e5336 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Request method failure captured: Jul 19 11:12:05.504592 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: request: GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67 HTTP/1.1 Jul 19 11:12:05.504592 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept: application/json Jul 19 11:12:05.504592 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept-Encoding: identity Jul 19 11:12:05.504592 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Connection: close Jul 19 11:12:05.504592 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Length: 0 Jul 19 11:12:05.504592 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Type: application/json Jul 19 11:12:05.504592 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Host: 10.222.0.33 Jul 19 11:12:05.504592 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Openstack-System-Scope: None Jul 19 11:12:05.504592 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: User-Agent: python-urllib3/1.26.9 Jul 19 11:12:05.504592 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Auth-Token: gAAAAABi1o_HYpL7r_umyWTTKsnaQkfxgRQPrCVzXYV3YxPvpHKAUsKkXxPNkPyHBhEDbahQjtwakXlDpqARj9YKw3tFYmG0gCU4SGJZDCiqyWHloRDABrhK0mtm6JR4S49vBX-Zeg1oz_jYGWaYn6kU0GfIXikwtQO5JQ3VMjNcjqJ15uYlE9Q Jul 19 11:12:05.504592 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Id: None Jul 19 11:12:05.504592 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Name: None Jul 19 11:12:05.504592 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Identity-Status: Confirmed Jul 19 11:12:05.504592 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Is-Admin-Project: True Jul 19 11:12:05.504592 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Id: default Jul 19 11:12:05.504592 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Name: Default Jul 19 11:12:05.504592 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Id: ef534f1d06ea415d98857ec7f9e5fb97 Jul 19 11:12:05.504592 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Name: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:12:05.504592 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Role: member,reader Jul 19 11:12:05.504592 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Roles: member,reader Jul 19 11:12:05.506037 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:12:05.506037 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:12:05.506037 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Id: ef534f1d06ea415d98857ec7f9e5fb97 Jul 19 11:12:05.506037 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:12:05.506037 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User: tempest-AttachVolumeNegativeTest-434278602-project Jul 19 11:12:05.506037 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Id: default Jul 19 11:12:05.506037 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Name: Default Jul 19 11:12:05.506037 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Id: b8f2a78f3a3e4b8e8937e3bc39524690 Jul 19 11:12:05.506037 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Name: tempest-AttachVolumeNegativeTest-434278602-project Jul 19 11:12:05.506037 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: method: > Jul 19 11:12:05.506037 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: exception: Instance e9cecc8a-4c06-454a-91b6-0b735a222e67 could not be found. Jul 19 11:12:05.506037 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: action_args: {'id': 'e9cecc8a-4c06-454a-91b6-0b735a222e67'} Jul 19 11:12:05.506775 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:12:05.506775 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:05.506775 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:12:05.506775 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:12:05.506775 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:12:05.506775 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:12:05.506775 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:12:05.506775 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:12:05.506775 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:12:05.506775 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:12:05.506775 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e9cecc8a-4c06-454a-91b6-0b735a222e67 could not be found. Jul 19 11:12:05.506775 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:05.506775 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:05.506775 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:05.506775 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:05.506775 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:05.506775 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:05.506775 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:12:05.506775 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:12:05.506775 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:12:05.507957 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:12:05.507957 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 19 11:12:05.507957 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 19 11:12:05.507957 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:12:05.507957 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:12:05.507957 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:12:05.507957 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:12:05.507957 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e9cecc8a-4c06-454a-91b6-0b735a222e67 could not be found. Jul 19 11:12:05.507957 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:05.507957 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:05.507957 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:05.507957 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:05.507957 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:05.507957 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:05.507957 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:12:05.507957 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:12:05.507957 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e9cecc8a-4c06-454a-91b6-0b735a222e67 could not be found. Jul 19 11:12:05.507957 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:05.509126 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-f61b6c58-4d70-428f-82c0-fab4916e5336 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Returning 404 to user: Instance e9cecc8a-4c06-454a-91b6-0b735a222e67 could not be found. {{(pid=117849) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:12:05.509126 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-f61b6c58-4d70-428f-82c0-fab4916e5336 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67" status: 404 len: 111 microversion: 2.1 time: 0.038867 Jul 19 11:12:05.509126 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 583/1173] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:12:05 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67 => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:12:05.516228 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-c6f2e347-a94f-4513-88ee-6cdf3b4b1e67 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '.version_select at 0x7f04aa4041f0>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:05.539854 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-c6f2e347-a94f-4513-88ee-6cdf3b4b1e67 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-609097279" status: 202 len: 0 microversion: 2.1 time: 0.028343 Jul 19 11:12:05.541040 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 591/1174] 10.222.0.33 () {58 vars in 1217 bytes} [Tue Jul 19 11:12:05 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-609097279 => generated 0 bytes in 30 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 19 11:12:07.374310 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-af93626a-f8e0-488d-b8a6-a33889fe6cdf tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:07.374796 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-af93626a-f8e0-488d-b8a6-a33889fe6cdf tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:07.381619 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-af93626a-f8e0-488d-b8a6-a33889fe6cdf tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:07.382071 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-af93626a-f8e0-488d-b8a6-a33889fe6cdf tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:07.382848 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-af93626a-f8e0-488d-b8a6-a33889fe6cdf tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:07.383285 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-af93626a-f8e0-488d-b8a6-a33889fe6cdf tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:07.397395 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.wsgi [None req-af93626a-f8e0-488d-b8a6-a33889fe6cdf tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] HTTP exception thrown: Instance e9cecc8a-4c06-454a-91b6-0b735a222e67 could not be found. Jul 19 11:12:07.399174 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-af93626a-f8e0-488d-b8a6-a33889fe6cdf tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Request method failure captured: Jul 19 11:12:07.399174 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: request: DELETE /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67 HTTP/1.1 Jul 19 11:12:07.399174 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept: application/json Jul 19 11:12:07.399174 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept-Encoding: identity Jul 19 11:12:07.399174 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Connection: close Jul 19 11:12:07.399174 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Length: 0 Jul 19 11:12:07.399174 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Type: application/json Jul 19 11:12:07.399174 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Host: 10.222.0.33 Jul 19 11:12:07.399174 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Openstack-System-Scope: None Jul 19 11:12:07.399174 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: User-Agent: python-urllib3/1.26.9 Jul 19 11:12:07.399174 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Auth-Token: gAAAAABi1o_HYpL7r_umyWTTKsnaQkfxgRQPrCVzXYV3YxPvpHKAUsKkXxPNkPyHBhEDbahQjtwakXlDpqARj9YKw3tFYmG0gCU4SGJZDCiqyWHloRDABrhK0mtm6JR4S49vBX-Zeg1oz_jYGWaYn6kU0GfIXikwtQO5JQ3VMjNcjqJ15uYlE9Q Jul 19 11:12:07.399174 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Id: None Jul 19 11:12:07.399174 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Name: None Jul 19 11:12:07.399174 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Identity-Status: Confirmed Jul 19 11:12:07.399174 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Is-Admin-Project: True Jul 19 11:12:07.399174 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Id: default Jul 19 11:12:07.399174 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Name: Default Jul 19 11:12:07.399174 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Id: ef534f1d06ea415d98857ec7f9e5fb97 Jul 19 11:12:07.399174 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Name: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:12:07.399174 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Role: member,reader Jul 19 11:12:07.399174 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Roles: member,reader Jul 19 11:12:07.400724 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:12:07.400724 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:12:07.400724 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Id: ef534f1d06ea415d98857ec7f9e5fb97 Jul 19 11:12:07.400724 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:12:07.400724 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User: tempest-AttachVolumeNegativeTest-434278602-project Jul 19 11:12:07.400724 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Id: default Jul 19 11:12:07.400724 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Name: Default Jul 19 11:12:07.400724 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Id: b8f2a78f3a3e4b8e8937e3bc39524690 Jul 19 11:12:07.400724 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Name: tempest-AttachVolumeNegativeTest-434278602-project Jul 19 11:12:07.400724 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: method: > Jul 19 11:12:07.400724 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: exception: Instance e9cecc8a-4c06-454a-91b6-0b735a222e67 could not be found. Jul 19 11:12:07.400724 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: action_args: {'id': 'e9cecc8a-4c06-454a-91b6-0b735a222e67'} Jul 19 11:12:07.401644 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:12:07.401644 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:07.401644 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:12:07.401644 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:12:07.401644 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:12:07.401644 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:12:07.401644 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:12:07.401644 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:12:07.401644 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:12:07.401644 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:12:07.401644 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e9cecc8a-4c06-454a-91b6-0b735a222e67 could not be found. Jul 19 11:12:07.401644 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:07.401644 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:07.401644 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:07.401644 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:07.401644 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:07.401644 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:07.401644 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:12:07.401644 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:12:07.401644 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:12:07.402984 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:12:07.402984 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 19 11:12:07.402984 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 19 11:12:07.402984 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 19 11:12:07.402984 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 19 11:12:07.402984 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:12:07.402984 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:12:07.402984 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:12:07.402984 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:12:07.402984 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e9cecc8a-4c06-454a-91b6-0b735a222e67 could not be found. Jul 19 11:12:07.402984 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:07.402984 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:07.402984 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:07.402984 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:07.402984 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:07.402984 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:07.402984 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:12:07.402984 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:12:07.402984 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e9cecc8a-4c06-454a-91b6-0b735a222e67 could not be found. Jul 19 11:12:07.402984 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:07.404272 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-af93626a-f8e0-488d-b8a6-a33889fe6cdf tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Returning 404 to user: Instance e9cecc8a-4c06-454a-91b6-0b735a222e67 could not be found. {{(pid=117849) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:12:07.404272 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-af93626a-f8e0-488d-b8a6-a33889fe6cdf tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "DELETE /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67" status: 404 len: 111 microversion: 2.1 time: 0.030037 Jul 19 11:12:07.404272 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 584/1175] 10.222.0.33 () {58 vars in 1238 bytes} [Tue Jul 19 11:12:07 2022] DELETE /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:12:07.410504 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-1a813b12-28d2-4a54-bf81-07d2b2a4e4f2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:07.411227 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-1a813b12-28d2-4a54-bf81-07d2b2a4e4f2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: e9cecc8a-4c06-454a-91b6-0b735a222e67] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:07.417409 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1a813b12-28d2-4a54-bf81-07d2b2a4e4f2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:07.417889 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1a813b12-28d2-4a54-bf81-07d2b2a4e4f2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:07.418649 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1a813b12-28d2-4a54-bf81-07d2b2a4e4f2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:07.419081 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1a813b12-28d2-4a54-bf81-07d2b2a4e4f2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:07.434170 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.wsgi [None req-1a813b12-28d2-4a54-bf81-07d2b2a4e4f2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] HTTP exception thrown: Instance e9cecc8a-4c06-454a-91b6-0b735a222e67 could not be found. Jul 19 11:12:07.437519 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-1a813b12-28d2-4a54-bf81-07d2b2a4e4f2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Request method failure captured: Jul 19 11:12:07.437519 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: request: GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67 HTTP/1.1 Jul 19 11:12:07.437519 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept: application/json Jul 19 11:12:07.437519 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept-Encoding: identity Jul 19 11:12:07.437519 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Connection: close Jul 19 11:12:07.437519 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Length: 0 Jul 19 11:12:07.437519 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Type: application/json Jul 19 11:12:07.437519 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Host: 10.222.0.33 Jul 19 11:12:07.437519 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Openstack-System-Scope: None Jul 19 11:12:07.437519 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: User-Agent: python-urllib3/1.26.9 Jul 19 11:12:07.437519 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Auth-Token: gAAAAABi1o_HYpL7r_umyWTTKsnaQkfxgRQPrCVzXYV3YxPvpHKAUsKkXxPNkPyHBhEDbahQjtwakXlDpqARj9YKw3tFYmG0gCU4SGJZDCiqyWHloRDABrhK0mtm6JR4S49vBX-Zeg1oz_jYGWaYn6kU0GfIXikwtQO5JQ3VMjNcjqJ15uYlE9Q Jul 19 11:12:07.437519 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Id: None Jul 19 11:12:07.437519 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Name: None Jul 19 11:12:07.437519 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Identity-Status: Confirmed Jul 19 11:12:07.437519 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Is-Admin-Project: True Jul 19 11:12:07.437519 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Id: default Jul 19 11:12:07.437519 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Name: Default Jul 19 11:12:07.437519 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Id: ef534f1d06ea415d98857ec7f9e5fb97 Jul 19 11:12:07.437519 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Name: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:12:07.437519 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Role: member,reader Jul 19 11:12:07.437519 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Roles: member,reader Jul 19 11:12:07.438892 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:12:07.438892 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:12:07.438892 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Id: ef534f1d06ea415d98857ec7f9e5fb97 Jul 19 11:12:07.438892 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:12:07.438892 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User: tempest-AttachVolumeNegativeTest-434278602-project Jul 19 11:12:07.438892 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Id: default Jul 19 11:12:07.438892 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Name: Default Jul 19 11:12:07.438892 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Id: b8f2a78f3a3e4b8e8937e3bc39524690 Jul 19 11:12:07.438892 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Name: tempest-AttachVolumeNegativeTest-434278602-project Jul 19 11:12:07.438892 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: method: > Jul 19 11:12:07.438892 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: exception: Instance e9cecc8a-4c06-454a-91b6-0b735a222e67 could not be found. Jul 19 11:12:07.438892 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: action_args: {'id': 'e9cecc8a-4c06-454a-91b6-0b735a222e67'} Jul 19 11:12:07.441355 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:12:07.441355 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:07.441355 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:12:07.441355 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:12:07.441355 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:12:07.441355 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:12:07.441355 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:12:07.441355 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:12:07.441355 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:12:07.441355 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:12:07.441355 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e9cecc8a-4c06-454a-91b6-0b735a222e67 could not be found. Jul 19 11:12:07.441355 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:07.441355 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:07.441355 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:07.441355 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:07.441355 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:07.441355 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:07.441355 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:12:07.441355 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:12:07.441355 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:12:07.442841 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:12:07.442841 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 19 11:12:07.442841 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 19 11:12:07.442841 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:12:07.442841 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:12:07.442841 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:12:07.442841 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:12:07.442841 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e9cecc8a-4c06-454a-91b6-0b735a222e67 could not be found. Jul 19 11:12:07.442841 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:07.442841 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:07.442841 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:07.442841 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:07.442841 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:07.442841 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:07.442841 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:12:07.442841 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:12:07.442841 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e9cecc8a-4c06-454a-91b6-0b735a222e67 could not be found. Jul 19 11:12:07.442841 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:07.443941 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-1a813b12-28d2-4a54-bf81-07d2b2a4e4f2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Returning 404 to user: Instance e9cecc8a-4c06-454a-91b6-0b735a222e67 could not be found. {{(pid=117848) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:12:07.443941 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-1a813b12-28d2-4a54-bf81-07d2b2a4e4f2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67" status: 404 len: 111 microversion: 2.1 time: 0.033585 Jul 19 11:12:07.443941 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 592/1176] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:12:07 2022] GET /compute/v2.1/servers/e9cecc8a-4c06-454a-91b6-0b735a222e67 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:12:08.606520 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-9be8b38b-2f13-4e63-8d3d-d9ada3cf8ef4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:08.607099 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-9be8b38b-2f13-4e63-8d3d-d9ada3cf8ef4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 53545a7c-c683-49d6-941d-4445f2b09379] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:08.613671 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9be8b38b-2f13-4e63-8d3d-d9ada3cf8ef4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:08.614196 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9be8b38b-2f13-4e63-8d3d-d9ada3cf8ef4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:08.615071 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9be8b38b-2f13-4e63-8d3d-d9ada3cf8ef4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:08.615592 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-9be8b38b-2f13-4e63-8d3d-d9ada3cf8ef4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:08.629713 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.wsgi [None req-9be8b38b-2f13-4e63-8d3d-d9ada3cf8ef4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] HTTP exception thrown: Instance 53545a7c-c683-49d6-941d-4445f2b09379 could not be found. Jul 19 11:12:08.631331 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-9be8b38b-2f13-4e63-8d3d-d9ada3cf8ef4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Request method failure captured: Jul 19 11:12:08.631331 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: request: DELETE /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379 HTTP/1.1 Jul 19 11:12:08.631331 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept: application/json Jul 19 11:12:08.631331 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept-Encoding: identity Jul 19 11:12:08.631331 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Connection: close Jul 19 11:12:08.631331 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Length: 0 Jul 19 11:12:08.631331 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Type: application/json Jul 19 11:12:08.631331 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Host: 10.222.0.33 Jul 19 11:12:08.631331 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Openstack-System-Scope: None Jul 19 11:12:08.631331 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: User-Agent: python-urllib3/1.26.9 Jul 19 11:12:08.631331 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Auth-Token: gAAAAABi1o_HYpL7r_umyWTTKsnaQkfxgRQPrCVzXYV3YxPvpHKAUsKkXxPNkPyHBhEDbahQjtwakXlDpqARj9YKw3tFYmG0gCU4SGJZDCiqyWHloRDABrhK0mtm6JR4S49vBX-Zeg1oz_jYGWaYn6kU0GfIXikwtQO5JQ3VMjNcjqJ15uYlE9Q Jul 19 11:12:08.631331 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Id: None Jul 19 11:12:08.631331 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Name: None Jul 19 11:12:08.631331 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Identity-Status: Confirmed Jul 19 11:12:08.631331 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Is-Admin-Project: True Jul 19 11:12:08.631331 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Id: default Jul 19 11:12:08.631331 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Name: Default Jul 19 11:12:08.631331 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Id: ef534f1d06ea415d98857ec7f9e5fb97 Jul 19 11:12:08.631331 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Name: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:12:08.631331 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Role: member,reader Jul 19 11:12:08.631331 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Roles: member,reader Jul 19 11:12:08.633007 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:12:08.633007 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:12:08.633007 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Id: ef534f1d06ea415d98857ec7f9e5fb97 Jul 19 11:12:08.633007 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:12:08.633007 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User: tempest-AttachVolumeNegativeTest-434278602-project Jul 19 11:12:08.633007 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Id: default Jul 19 11:12:08.633007 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Name: Default Jul 19 11:12:08.633007 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Id: b8f2a78f3a3e4b8e8937e3bc39524690 Jul 19 11:12:08.633007 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Name: tempest-AttachVolumeNegativeTest-434278602-project Jul 19 11:12:08.633007 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: method: > Jul 19 11:12:08.633007 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: exception: Instance 53545a7c-c683-49d6-941d-4445f2b09379 could not be found. Jul 19 11:12:08.633007 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: action_args: {'id': '53545a7c-c683-49d6-941d-4445f2b09379'} Jul 19 11:12:08.633858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:12:08.633858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:08.633858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:12:08.633858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:12:08.633858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:12:08.633858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:12:08.633858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:12:08.633858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:12:08.633858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:12:08.633858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:12:08.633858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 53545a7c-c683-49d6-941d-4445f2b09379 could not be found. Jul 19 11:12:08.633858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:08.633858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:08.633858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:08.633858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:08.633858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:08.633858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:08.633858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:12:08.633858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:12:08.633858 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:12:08.635217 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:12:08.635217 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 19 11:12:08.635217 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 19 11:12:08.635217 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 19 11:12:08.635217 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 19 11:12:08.635217 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:12:08.635217 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:12:08.635217 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:12:08.635217 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:12:08.635217 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 53545a7c-c683-49d6-941d-4445f2b09379 could not be found. Jul 19 11:12:08.635217 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:08.635217 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:08.635217 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:08.635217 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:08.635217 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:08.635217 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:08.635217 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:12:08.635217 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:12:08.635217 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 53545a7c-c683-49d6-941d-4445f2b09379 could not be found. Jul 19 11:12:08.635217 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:08.636615 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-9be8b38b-2f13-4e63-8d3d-d9ada3cf8ef4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Returning 404 to user: Instance 53545a7c-c683-49d6-941d-4445f2b09379 could not be found. {{(pid=117849) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:12:08.636615 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-9be8b38b-2f13-4e63-8d3d-d9ada3cf8ef4 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "DELETE /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379" status: 404 len: 111 microversion: 2.1 time: 0.030570 Jul 19 11:12:08.636615 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 585/1177] 10.222.0.33 () {58 vars in 1238 bytes} [Tue Jul 19 11:12:08 2022] DELETE /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:12:08.641996 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-0f877ad7-a484-41bd-9449-11a78a7b11c9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:08.642602 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-0f877ad7-a484-41bd-9449-11a78a7b11c9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 53545a7c-c683-49d6-941d-4445f2b09379] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:08.649169 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0f877ad7-a484-41bd-9449-11a78a7b11c9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:08.649614 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0f877ad7-a484-41bd-9449-11a78a7b11c9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:08.650374 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0f877ad7-a484-41bd-9449-11a78a7b11c9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:08.650803 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0f877ad7-a484-41bd-9449-11a78a7b11c9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:08.664442 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.wsgi [None req-0f877ad7-a484-41bd-9449-11a78a7b11c9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] HTTP exception thrown: Instance 53545a7c-c683-49d6-941d-4445f2b09379 could not be found. Jul 19 11:12:08.665983 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-0f877ad7-a484-41bd-9449-11a78a7b11c9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Request method failure captured: Jul 19 11:12:08.665983 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: request: GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379 HTTP/1.1 Jul 19 11:12:08.665983 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept: application/json Jul 19 11:12:08.665983 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept-Encoding: identity Jul 19 11:12:08.665983 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Connection: close Jul 19 11:12:08.665983 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Length: 0 Jul 19 11:12:08.665983 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Type: application/json Jul 19 11:12:08.665983 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Host: 10.222.0.33 Jul 19 11:12:08.665983 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Openstack-System-Scope: None Jul 19 11:12:08.665983 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: User-Agent: python-urllib3/1.26.9 Jul 19 11:12:08.665983 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Auth-Token: gAAAAABi1o_HYpL7r_umyWTTKsnaQkfxgRQPrCVzXYV3YxPvpHKAUsKkXxPNkPyHBhEDbahQjtwakXlDpqARj9YKw3tFYmG0gCU4SGJZDCiqyWHloRDABrhK0mtm6JR4S49vBX-Zeg1oz_jYGWaYn6kU0GfIXikwtQO5JQ3VMjNcjqJ15uYlE9Q Jul 19 11:12:08.665983 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Id: None Jul 19 11:12:08.665983 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Name: None Jul 19 11:12:08.665983 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Identity-Status: Confirmed Jul 19 11:12:08.665983 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Is-Admin-Project: True Jul 19 11:12:08.665983 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Id: default Jul 19 11:12:08.665983 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Name: Default Jul 19 11:12:08.665983 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Id: ef534f1d06ea415d98857ec7f9e5fb97 Jul 19 11:12:08.665983 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Name: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:12:08.665983 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Role: member,reader Jul 19 11:12:08.665983 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Roles: member,reader Jul 19 11:12:08.667541 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:12:08.667541 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:12:08.667541 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Id: ef534f1d06ea415d98857ec7f9e5fb97 Jul 19 11:12:08.667541 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:12:08.667541 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User: tempest-AttachVolumeNegativeTest-434278602-project Jul 19 11:12:08.667541 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Id: default Jul 19 11:12:08.667541 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Name: Default Jul 19 11:12:08.667541 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Id: b8f2a78f3a3e4b8e8937e3bc39524690 Jul 19 11:12:08.667541 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Name: tempest-AttachVolumeNegativeTest-434278602-project Jul 19 11:12:08.667541 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: method: > Jul 19 11:12:08.667541 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: exception: Instance 53545a7c-c683-49d6-941d-4445f2b09379 could not be found. Jul 19 11:12:08.667541 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: action_args: {'id': '53545a7c-c683-49d6-941d-4445f2b09379'} Jul 19 11:12:08.668437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:12:08.668437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:08.668437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:12:08.668437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:12:08.668437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:12:08.668437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:12:08.668437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:12:08.668437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:12:08.668437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:12:08.668437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:12:08.668437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 53545a7c-c683-49d6-941d-4445f2b09379 could not be found. Jul 19 11:12:08.668437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:08.668437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:08.668437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:08.668437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:08.668437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:08.668437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:08.668437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:12:08.668437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:12:08.668437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:12:08.669973 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:12:08.669973 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 19 11:12:08.669973 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 19 11:12:08.669973 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:12:08.669973 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:12:08.669973 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:12:08.669973 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:12:08.669973 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 53545a7c-c683-49d6-941d-4445f2b09379 could not be found. Jul 19 11:12:08.669973 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:08.669973 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:08.669973 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:08.669973 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:08.669973 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:08.669973 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:08.669973 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:12:08.669973 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:12:08.669973 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 53545a7c-c683-49d6-941d-4445f2b09379 could not be found. Jul 19 11:12:08.669973 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:08.671532 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-0f877ad7-a484-41bd-9449-11a78a7b11c9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Returning 404 to user: Instance 53545a7c-c683-49d6-941d-4445f2b09379 could not be found. {{(pid=117848) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:12:08.671532 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-0f877ad7-a484-41bd-9449-11a78a7b11c9 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379" status: 404 len: 111 microversion: 2.1 time: 0.029191 Jul 19 11:12:08.671532 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 593/1178] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:12:08 2022] GET /compute/v2.1/servers/53545a7c-c683-49d6-941d-4445f2b09379 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:12:08.676310 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-a4949b00-a523-4a13-967e-44988da420a2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:08.676941 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-a4949b00-a523-4a13-967e-44988da420a2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 988f14d7-6354-4388-a51c-b4df9749c671] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:08.682997 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a4949b00-a523-4a13-967e-44988da420a2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:08.683441 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a4949b00-a523-4a13-967e-44988da420a2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:08.684206 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a4949b00-a523-4a13-967e-44988da420a2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:08.684659 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a4949b00-a523-4a13-967e-44988da420a2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:08.698249 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.wsgi [None req-a4949b00-a523-4a13-967e-44988da420a2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] HTTP exception thrown: Instance 988f14d7-6354-4388-a51c-b4df9749c671 could not be found. Jul 19 11:12:08.699795 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-a4949b00-a523-4a13-967e-44988da420a2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Request method failure captured: Jul 19 11:12:08.699795 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: request: DELETE /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671 HTTP/1.1 Jul 19 11:12:08.699795 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept: application/json Jul 19 11:12:08.699795 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept-Encoding: identity Jul 19 11:12:08.699795 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Connection: close Jul 19 11:12:08.699795 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Length: 0 Jul 19 11:12:08.699795 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Type: application/json Jul 19 11:12:08.699795 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Host: 10.222.0.33 Jul 19 11:12:08.699795 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Openstack-System-Scope: None Jul 19 11:12:08.699795 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: User-Agent: python-urllib3/1.26.9 Jul 19 11:12:08.699795 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Auth-Token: gAAAAABi1o_HYpL7r_umyWTTKsnaQkfxgRQPrCVzXYV3YxPvpHKAUsKkXxPNkPyHBhEDbahQjtwakXlDpqARj9YKw3tFYmG0gCU4SGJZDCiqyWHloRDABrhK0mtm6JR4S49vBX-Zeg1oz_jYGWaYn6kU0GfIXikwtQO5JQ3VMjNcjqJ15uYlE9Q Jul 19 11:12:08.699795 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Id: None Jul 19 11:12:08.699795 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Name: None Jul 19 11:12:08.699795 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Identity-Status: Confirmed Jul 19 11:12:08.699795 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Is-Admin-Project: True Jul 19 11:12:08.699795 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Id: default Jul 19 11:12:08.699795 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Name: Default Jul 19 11:12:08.699795 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Id: ef534f1d06ea415d98857ec7f9e5fb97 Jul 19 11:12:08.699795 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Name: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:12:08.699795 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Role: member,reader Jul 19 11:12:08.699795 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Roles: member,reader Jul 19 11:12:08.701471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:12:08.701471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:12:08.701471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Id: ef534f1d06ea415d98857ec7f9e5fb97 Jul 19 11:12:08.701471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:12:08.701471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User: tempest-AttachVolumeNegativeTest-434278602-project Jul 19 11:12:08.701471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Id: default Jul 19 11:12:08.701471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Name: Default Jul 19 11:12:08.701471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Id: b8f2a78f3a3e4b8e8937e3bc39524690 Jul 19 11:12:08.701471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Name: tempest-AttachVolumeNegativeTest-434278602-project Jul 19 11:12:08.701471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: method: > Jul 19 11:12:08.701471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: exception: Instance 988f14d7-6354-4388-a51c-b4df9749c671 could not be found. Jul 19 11:12:08.701471 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: action_args: {'id': '988f14d7-6354-4388-a51c-b4df9749c671'} Jul 19 11:12:08.702378 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:12:08.702378 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:08.702378 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:12:08.702378 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:12:08.702378 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:12:08.702378 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:12:08.702378 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:12:08.702378 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:12:08.702378 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:12:08.702378 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:12:08.702378 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 988f14d7-6354-4388-a51c-b4df9749c671 could not be found. Jul 19 11:12:08.702378 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:08.702378 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:08.702378 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:08.702378 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:08.702378 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:08.702378 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:08.702378 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:12:08.702378 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:12:08.702378 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:12:08.703846 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:12:08.703846 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 19 11:12:08.703846 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 19 11:12:08.703846 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 19 11:12:08.703846 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 19 11:12:08.703846 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:12:08.703846 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:12:08.703846 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:12:08.703846 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:12:08.703846 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 988f14d7-6354-4388-a51c-b4df9749c671 could not be found. Jul 19 11:12:08.703846 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:08.703846 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:08.703846 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:08.703846 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:08.703846 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:08.703846 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:08.703846 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:12:08.703846 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:12:08.703846 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 988f14d7-6354-4388-a51c-b4df9749c671 could not be found. Jul 19 11:12:08.703846 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:08.705274 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-a4949b00-a523-4a13-967e-44988da420a2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Returning 404 to user: Instance 988f14d7-6354-4388-a51c-b4df9749c671 could not be found. {{(pid=117849) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:12:08.705274 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-a4949b00-a523-4a13-967e-44988da420a2 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "DELETE /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671" status: 404 len: 111 microversion: 2.1 time: 0.029280 Jul 19 11:12:08.705274 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 586/1179] 10.222.0.33 () {58 vars in 1238 bytes} [Tue Jul 19 11:12:08 2022] DELETE /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:12:08.714630 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-23169d6d-dffa-4f16-b0a2-89b858f3869d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:08.715659 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-23169d6d-dffa-4f16-b0a2-89b858f3869d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 988f14d7-6354-4388-a51c-b4df9749c671] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:08.723265 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-23169d6d-dffa-4f16-b0a2-89b858f3869d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:08.723917 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-23169d6d-dffa-4f16-b0a2-89b858f3869d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:08.725469 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-23169d6d-dffa-4f16-b0a2-89b858f3869d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:08.726100 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-23169d6d-dffa-4f16-b0a2-89b858f3869d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:08.743196 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.wsgi [None req-23169d6d-dffa-4f16-b0a2-89b858f3869d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] HTTP exception thrown: Instance 988f14d7-6354-4388-a51c-b4df9749c671 could not be found. Jul 19 11:12:08.744798 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-23169d6d-dffa-4f16-b0a2-89b858f3869d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Request method failure captured: Jul 19 11:12:08.744798 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: request: GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671 HTTP/1.1 Jul 19 11:12:08.744798 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept: application/json Jul 19 11:12:08.744798 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept-Encoding: identity Jul 19 11:12:08.744798 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Connection: close Jul 19 11:12:08.744798 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Length: 0 Jul 19 11:12:08.744798 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Type: application/json Jul 19 11:12:08.744798 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Host: 10.222.0.33 Jul 19 11:12:08.744798 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Openstack-System-Scope: None Jul 19 11:12:08.744798 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: User-Agent: python-urllib3/1.26.9 Jul 19 11:12:08.744798 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Auth-Token: gAAAAABi1o_HYpL7r_umyWTTKsnaQkfxgRQPrCVzXYV3YxPvpHKAUsKkXxPNkPyHBhEDbahQjtwakXlDpqARj9YKw3tFYmG0gCU4SGJZDCiqyWHloRDABrhK0mtm6JR4S49vBX-Zeg1oz_jYGWaYn6kU0GfIXikwtQO5JQ3VMjNcjqJ15uYlE9Q Jul 19 11:12:08.744798 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Id: None Jul 19 11:12:08.744798 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Name: None Jul 19 11:12:08.744798 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Identity-Status: Confirmed Jul 19 11:12:08.744798 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Is-Admin-Project: True Jul 19 11:12:08.744798 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Id: default Jul 19 11:12:08.744798 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Name: Default Jul 19 11:12:08.744798 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Id: ef534f1d06ea415d98857ec7f9e5fb97 Jul 19 11:12:08.744798 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Name: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:12:08.744798 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Role: member,reader Jul 19 11:12:08.744798 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Roles: member,reader Jul 19 11:12:08.746521 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:12:08.746521 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:12:08.746521 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Id: ef534f1d06ea415d98857ec7f9e5fb97 Jul 19 11:12:08.746521 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:12:08.746521 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User: tempest-AttachVolumeNegativeTest-434278602-project Jul 19 11:12:08.746521 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Id: default Jul 19 11:12:08.746521 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Name: Default Jul 19 11:12:08.746521 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Id: b8f2a78f3a3e4b8e8937e3bc39524690 Jul 19 11:12:08.746521 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Name: tempest-AttachVolumeNegativeTest-434278602-project Jul 19 11:12:08.746521 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: method: > Jul 19 11:12:08.746521 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: exception: Instance 988f14d7-6354-4388-a51c-b4df9749c671 could not be found. Jul 19 11:12:08.746521 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: action_args: {'id': '988f14d7-6354-4388-a51c-b4df9749c671'} Jul 19 11:12:08.750850 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:12:08.750850 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:08.750850 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:12:08.750850 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:12:08.750850 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:12:08.750850 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:12:08.750850 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:12:08.750850 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:12:08.750850 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:12:08.750850 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:12:08.750850 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 988f14d7-6354-4388-a51c-b4df9749c671 could not be found. Jul 19 11:12:08.750850 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:08.750850 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:08.750850 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:08.750850 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:08.750850 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:08.750850 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:08.750850 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:12:08.750850 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:12:08.750850 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:12:08.752336 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:12:08.752336 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 19 11:12:08.752336 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 19 11:12:08.752336 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:12:08.752336 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:12:08.752336 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:12:08.752336 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:12:08.752336 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 988f14d7-6354-4388-a51c-b4df9749c671 could not be found. Jul 19 11:12:08.752336 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:08.752336 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:08.752336 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:08.752336 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:08.752336 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:08.752336 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:08.752336 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:12:08.752336 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:12:08.752336 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 988f14d7-6354-4388-a51c-b4df9749c671 could not be found. Jul 19 11:12:08.752336 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:08.754142 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-23169d6d-dffa-4f16-b0a2-89b858f3869d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Returning 404 to user: Instance 988f14d7-6354-4388-a51c-b4df9749c671 could not be found. {{(pid=117848) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:12:08.754142 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-23169d6d-dffa-4f16-b0a2-89b858f3869d tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671" status: 404 len: 111 microversion: 2.1 time: 0.036138 Jul 19 11:12:08.754142 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 594/1180] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:12:08 2022] GET /compute/v2.1/servers/988f14d7-6354-4388-a51c-b4df9749c671 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:12:09.906678 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-47b9b05a-0937-4d7f-aa65-a422f7816456 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:09.908079 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-47b9b05a-0937-4d7f-aa65-a422f7816456 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:09.926457 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-47b9b05a-0937-4d7f-aa65-a422f7816456 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:09.926910 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-47b9b05a-0937-4d7f-aa65-a422f7816456 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:09.927702 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-47b9b05a-0937-4d7f-aa65-a422f7816456 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:09.928135 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-47b9b05a-0937-4d7f-aa65-a422f7816456 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:09.954529 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.wsgi [None req-47b9b05a-0937-4d7f-aa65-a422f7816456 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] HTTP exception thrown: Instance 4c71f564-de63-4fcd-88ce-01ecba3d8372 could not be found. Jul 19 11:12:09.956244 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-47b9b05a-0937-4d7f-aa65-a422f7816456 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Request method failure captured: Jul 19 11:12:09.956244 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: request: DELETE /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 HTTP/1.1 Jul 19 11:12:09.956244 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept: application/json Jul 19 11:12:09.956244 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept-Encoding: identity Jul 19 11:12:09.956244 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Connection: close Jul 19 11:12:09.956244 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Length: 0 Jul 19 11:12:09.956244 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Type: application/json Jul 19 11:12:09.956244 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Host: 10.222.0.33 Jul 19 11:12:09.956244 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Openstack-System-Scope: None Jul 19 11:12:09.956244 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: User-Agent: python-urllib3/1.26.9 Jul 19 11:12:09.956244 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Auth-Token: gAAAAABi1o_HYpL7r_umyWTTKsnaQkfxgRQPrCVzXYV3YxPvpHKAUsKkXxPNkPyHBhEDbahQjtwakXlDpqARj9YKw3tFYmG0gCU4SGJZDCiqyWHloRDABrhK0mtm6JR4S49vBX-Zeg1oz_jYGWaYn6kU0GfIXikwtQO5JQ3VMjNcjqJ15uYlE9Q Jul 19 11:12:09.956244 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Id: None Jul 19 11:12:09.956244 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Name: None Jul 19 11:12:09.956244 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Identity-Status: Confirmed Jul 19 11:12:09.956244 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Is-Admin-Project: True Jul 19 11:12:09.956244 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Id: default Jul 19 11:12:09.956244 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Name: Default Jul 19 11:12:09.956244 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Id: ef534f1d06ea415d98857ec7f9e5fb97 Jul 19 11:12:09.956244 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Name: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:12:09.956244 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Role: member,reader Jul 19 11:12:09.956244 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Roles: member,reader Jul 19 11:12:09.957812 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:12:09.957812 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:12:09.957812 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Id: ef534f1d06ea415d98857ec7f9e5fb97 Jul 19 11:12:09.957812 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:12:09.957812 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User: tempest-AttachVolumeNegativeTest-434278602-project Jul 19 11:12:09.957812 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Id: default Jul 19 11:12:09.957812 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Name: Default Jul 19 11:12:09.957812 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Id: b8f2a78f3a3e4b8e8937e3bc39524690 Jul 19 11:12:09.957812 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Name: tempest-AttachVolumeNegativeTest-434278602-project Jul 19 11:12:09.957812 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: method: > Jul 19 11:12:09.957812 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: exception: Instance 4c71f564-de63-4fcd-88ce-01ecba3d8372 could not be found. Jul 19 11:12:09.957812 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: action_args: {'id': '4c71f564-de63-4fcd-88ce-01ecba3d8372'} Jul 19 11:12:09.958615 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:12:09.958615 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:09.958615 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:12:09.958615 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:12:09.958615 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:12:09.958615 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:12:09.958615 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:12:09.958615 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:12:09.958615 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:12:09.958615 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:12:09.958615 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4c71f564-de63-4fcd-88ce-01ecba3d8372 could not be found. Jul 19 11:12:09.958615 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:09.958615 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:09.958615 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:09.958615 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:09.958615 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:09.958615 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:09.958615 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:12:09.958615 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:12:09.958615 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:12:09.959867 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:12:09.959867 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 19 11:12:09.959867 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 19 11:12:09.959867 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 19 11:12:09.959867 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 19 11:12:09.959867 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:12:09.959867 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:12:09.959867 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:12:09.959867 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:12:09.959867 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4c71f564-de63-4fcd-88ce-01ecba3d8372 could not be found. Jul 19 11:12:09.959867 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:09.959867 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:09.959867 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:09.959867 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:09.959867 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:09.959867 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:09.959867 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:12:09.959867 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:12:09.959867 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4c71f564-de63-4fcd-88ce-01ecba3d8372 could not be found. Jul 19 11:12:09.959867 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:09.961952 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-47b9b05a-0937-4d7f-aa65-a422f7816456 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Returning 404 to user: Instance 4c71f564-de63-4fcd-88ce-01ecba3d8372 could not be found. {{(pid=117849) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:12:09.962800 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-47b9b05a-0937-4d7f-aa65-a422f7816456 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "DELETE /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 404 len: 111 microversion: 2.1 time: 0.066625 Jul 19 11:12:09.963506 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 587/1181] 10.222.0.33 () {58 vars in 1238 bytes} [Tue Jul 19 11:12:09 2022] DELETE /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 111 bytes in 70 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:12:09.971972 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-fbcac71a-1835-410f-b7ee-0f993d154776 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:09.972584 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-fbcac71a-1835-410f-b7ee-0f993d154776 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] [instance: 4c71f564-de63-4fcd-88ce-01ecba3d8372] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:09.978917 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fbcac71a-1835-410f-b7ee-0f993d154776 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:09.979356 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fbcac71a-1835-410f-b7ee-0f993d154776 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:09.980127 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fbcac71a-1835-410f-b7ee-0f993d154776 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:09.980558 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fbcac71a-1835-410f-b7ee-0f993d154776 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:10.009264 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.wsgi [None req-fbcac71a-1835-410f-b7ee-0f993d154776 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] HTTP exception thrown: Instance 4c71f564-de63-4fcd-88ce-01ecba3d8372 could not be found. Jul 19 11:12:10.010937 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-fbcac71a-1835-410f-b7ee-0f993d154776 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Request method failure captured: Jul 19 11:12:10.010937 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: request: GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 HTTP/1.1 Jul 19 11:12:10.010937 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept: application/json Jul 19 11:12:10.010937 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept-Encoding: identity Jul 19 11:12:10.010937 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Connection: close Jul 19 11:12:10.010937 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Length: 0 Jul 19 11:12:10.010937 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Type: application/json Jul 19 11:12:10.010937 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Host: 10.222.0.33 Jul 19 11:12:10.010937 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Openstack-System-Scope: None Jul 19 11:12:10.010937 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: User-Agent: python-urllib3/1.26.9 Jul 19 11:12:10.010937 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Auth-Token: gAAAAABi1o_HYpL7r_umyWTTKsnaQkfxgRQPrCVzXYV3YxPvpHKAUsKkXxPNkPyHBhEDbahQjtwakXlDpqARj9YKw3tFYmG0gCU4SGJZDCiqyWHloRDABrhK0mtm6JR4S49vBX-Zeg1oz_jYGWaYn6kU0GfIXikwtQO5JQ3VMjNcjqJ15uYlE9Q Jul 19 11:12:10.010937 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Id: None Jul 19 11:12:10.010937 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Name: None Jul 19 11:12:10.010937 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Identity-Status: Confirmed Jul 19 11:12:10.010937 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Is-Admin-Project: True Jul 19 11:12:10.010937 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Id: default Jul 19 11:12:10.010937 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Name: Default Jul 19 11:12:10.010937 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Id: ef534f1d06ea415d98857ec7f9e5fb97 Jul 19 11:12:10.010937 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Name: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:12:10.010937 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Role: member,reader Jul 19 11:12:10.010937 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Roles: member,reader Jul 19 11:12:10.012223 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_ef534f1d06ea415d98857ec7f9e5fb97"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:12:10.012223 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:12:10.012223 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Id: ef534f1d06ea415d98857ec7f9e5fb97 Jul 19 11:12:10.012223 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-434278602 Jul 19 11:12:10.012223 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User: tempest-AttachVolumeNegativeTest-434278602-project Jul 19 11:12:10.012223 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Id: default Jul 19 11:12:10.012223 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Name: Default Jul 19 11:12:10.012223 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Id: b8f2a78f3a3e4b8e8937e3bc39524690 Jul 19 11:12:10.012223 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Name: tempest-AttachVolumeNegativeTest-434278602-project Jul 19 11:12:10.012223 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: method: > Jul 19 11:12:10.012223 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: exception: Instance 4c71f564-de63-4fcd-88ce-01ecba3d8372 could not be found. Jul 19 11:12:10.012223 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: action_args: {'id': '4c71f564-de63-4fcd-88ce-01ecba3d8372'} Jul 19 11:12:10.012986 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:12:10.012986 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:10.012986 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:12:10.012986 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:12:10.012986 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:12:10.012986 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:12:10.012986 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:12:10.012986 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:12:10.012986 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:12:10.012986 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:12:10.012986 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4c71f564-de63-4fcd-88ce-01ecba3d8372 could not be found. Jul 19 11:12:10.012986 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:10.012986 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:10.012986 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:10.012986 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:10.012986 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:10.012986 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:10.012986 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:12:10.012986 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:12:10.012986 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:12:10.014635 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:12:10.014635 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 19 11:12:10.014635 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 19 11:12:10.014635 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:12:10.014635 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:12:10.014635 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:12:10.014635 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:12:10.014635 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4c71f564-de63-4fcd-88ce-01ecba3d8372 could not be found. Jul 19 11:12:10.014635 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:10.014635 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:10.014635 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:10.014635 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:10.014635 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:10.014635 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:10.014635 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:12:10.014635 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:12:10.014635 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4c71f564-de63-4fcd-88ce-01ecba3d8372 could not be found. Jul 19 11:12:10.014635 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:10.015693 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-fbcac71a-1835-410f-b7ee-0f993d154776 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] Returning 404 to user: Instance 4c71f564-de63-4fcd-88ce-01ecba3d8372 could not be found. {{(pid=117848) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:12:10.015693 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-fbcac71a-1835-410f-b7ee-0f993d154776 tempest-AttachVolumeNegativeTest-434278602 tempest-AttachVolumeNegativeTest-434278602-project] 10.222.0.33 "GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372" status: 404 len: 111 microversion: 2.1 time: 0.044999 Jul 19 11:12:10.015693 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 595/1182] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:12:09 2022] GET /compute/v2.1/servers/4c71f564-de63-4fcd-88ce-01ecba3d8372 => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:12:10.068185 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-03a354be-bc28-47be-8f8c-19afb4216b69 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:10.069123 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-03a354be-bc28-47be-8f8c-19afb4216b69 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:10.071064 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_api:SELECT=99,nova_api:DELETE=1 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:12:10.081824 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-03a354be-bc28-47be-8f8c-19afb4216b69 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:10.082436 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-03a354be-bc28-47be-8f8c-19afb4216b69 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:10.083344 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-03a354be-bc28-47be-8f8c-19afb4216b69 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:10.083777 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-03a354be-bc28-47be-8f8c-19afb4216b69 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:10.088369 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=100 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:12:10.116814 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-03a354be-bc28-47be-8f8c-19afb4216b69 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052372 Jul 19 11:12:10.118210 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 588/1183] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:12:10 2022] GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:12:10.128208 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-427bc2fd-aca3-452c-8e54-52673155dd7b tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:10.129183 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-427bc2fd-aca3-452c-8e54-52673155dd7b tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:10.137013 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-427bc2fd-aca3-452c-8e54-52673155dd7b tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:10.137984 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-427bc2fd-aca3-452c-8e54-52673155dd7b tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:10.139025 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-427bc2fd-aca3-452c-8e54-52673155dd7b tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:10.139698 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-427bc2fd-aca3-452c-8e54-52673155dd7b tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:10.178025 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-427bc2fd-aca3-452c-8e54-52673155dd7b tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75/os-volume_attachments/d5e72a9b-47b0-49af-b136-1c4a4ae09617" status: 200 len: 194 microversion: 2.1 time: 0.054702 Jul 19 11:12:10.178953 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 596/1184] 10.222.0.33 () {58 vars in 1412 bytes} [Tue Jul 19 11:12:10 2022] GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75/os-volume_attachments/d5e72a9b-47b0-49af-b136-1c4a4ae09617 => generated 194 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:12:11.582937 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-b03679ce-b64d-40ef-8775-fed4696e7b19 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "854d21ee-4ea2-4b71-8a40-bd0678e1468c"}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:12:11.583996 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-b03679ce-b64d-40ef-8775-fed4696e7b19 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:11.590489 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b03679ce-b64d-40ef-8775-fed4696e7b19 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:11.590942 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b03679ce-b64d-40ef-8775-fed4696e7b19 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:11.591714 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b03679ce-b64d-40ef-8775-fed4696e7b19 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:11.592326 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b03679ce-b64d-40ef-8775-fed4696e7b19 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:11.963719 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-b03679ce-b64d-40ef-8775-fed4696e7b19 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "POST /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.383654 Jul 19 11:12:11.965494 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 589/1185] 10.222.0.33 () {60 vars in 1322 bytes} [Tue Jul 19 11:12:11 2022] POST /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75/os-volume_attachments => generated 194 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:12:19.540336 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-8eb8eba6-583f-471d-b1ca-f63d82865f2f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:19.541275 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-8eb8eba6-583f-471d-b1ca-f63d82865f2f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:19.547843 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8eb8eba6-583f-471d-b1ca-f63d82865f2f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:19.548288 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8eb8eba6-583f-471d-b1ca-f63d82865f2f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:19.549077 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8eb8eba6-583f-471d-b1ca-f63d82865f2f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:19.549507 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-8eb8eba6-583f-471d-b1ca-f63d82865f2f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:19.578607 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-8eb8eba6-583f-471d-b1ca-f63d82865f2f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.042216 Jul 19 11:12:19.579151 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 597/1186] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:12:19 2022] GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75/os-volume_attachments => generated 371 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:12:19.723616 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-723d0b1e-604c-4d0d-bb92-1f0042277cfe tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:19.724125 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-723d0b1e-604c-4d0d-bb92-1f0042277cfe tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:19.730386 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-723d0b1e-604c-4d0d-bb92-1f0042277cfe tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:19.730826 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-723d0b1e-604c-4d0d-bb92-1f0042277cfe tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:19.731575 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-723d0b1e-604c-4d0d-bb92-1f0042277cfe tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:19.732005 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-723d0b1e-604c-4d0d-bb92-1f0042277cfe tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:19.760426 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-723d0b1e-604c-4d0d-bb92-1f0042277cfe tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75/os-volume_attachments/d5e72a9b-47b0-49af-b136-1c4a4ae09617" status: 200 len: 194 microversion: 2.1 time: 0.042062 Jul 19 11:12:19.761037 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 590/1187] 10.222.0.33 () {58 vars in 1412 bytes} [Tue Jul 19 11:12:19 2022] GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75/os-volume_attachments/d5e72a9b-47b0-49af-b136-1c4a4ae09617 => generated 194 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:12:19.771141 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-177c8053-b77d-4f4e-be01-34a1a6fe3483 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:19.771633 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-177c8053-b77d-4f4e-be01-34a1a6fe3483 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:19.778332 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-177c8053-b77d-4f4e-be01-34a1a6fe3483 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:19.778775 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-177c8053-b77d-4f4e-be01-34a1a6fe3483 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:19.779536 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-177c8053-b77d-4f4e-be01-34a1a6fe3483 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:19.779971 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-177c8053-b77d-4f4e-be01-34a1a6fe3483 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:19.857567 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.objects.instance [None req-177c8053-b77d-4f4e-be01-34a1a6fe3483 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lazy-loading 'services' on Instance uuid 2c0cb519-166b-4f4d-ae3e-46309a6d7c75 {{(pid=117848) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 19 11:12:19.929589 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-177c8053-b77d-4f4e-be01-34a1a6fe3483 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "DELETE /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75/os-volume_attachments/d5e72a9b-47b0-49af-b136-1c4a4ae09617" status: 202 len: 0 microversion: 2.1 time: 0.162780 Jul 19 11:12:19.930524 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 598/1188] 10.222.0.33 () {58 vars in 1415 bytes} [Tue Jul 19 11:12:19 2022] DELETE /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75/os-volume_attachments/d5e72a9b-47b0-49af-b136-1c4a4ae09617 => generated 0 bytes in 164 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 19 11:12:22.093159 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-f4c8ca9c-646f-4b63-9de4-0d7034a325c4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:22.093637 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-f4c8ca9c-646f-4b63-9de4-0d7034a325c4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:22.100748 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f4c8ca9c-646f-4b63-9de4-0d7034a325c4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:22.101198 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f4c8ca9c-646f-4b63-9de4-0d7034a325c4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:22.101956 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f4c8ca9c-646f-4b63-9de4-0d7034a325c4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:22.102389 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-f4c8ca9c-646f-4b63-9de4-0d7034a325c4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:22.131747 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-f4c8ca9c-646f-4b63-9de4-0d7034a325c4 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75/os-volume_attachments/854d21ee-4ea2-4b71-8a40-bd0678e1468c" status: 200 len: 194 microversion: 2.1 time: 0.042564 Jul 19 11:12:22.132338 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 591/1189] 10.222.0.33 () {58 vars in 1412 bytes} [Tue Jul 19 11:12:22 2022] GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75/os-volume_attachments/854d21ee-4ea2-4b71-8a40-bd0678e1468c => generated 194 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:12:22.143018 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-66e610ba-f56e-4f64-ac35-507bf719eb60 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:22.144131 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-66e610ba-f56e-4f64-ac35-507bf719eb60 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:22.150788 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-66e610ba-f56e-4f64-ac35-507bf719eb60 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:22.151243 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-66e610ba-f56e-4f64-ac35-507bf719eb60 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:22.152004 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-66e610ba-f56e-4f64-ac35-507bf719eb60 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:22.152439 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-66e610ba-f56e-4f64-ac35-507bf719eb60 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:22.233783 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.objects.instance [None req-66e610ba-f56e-4f64-ac35-507bf719eb60 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lazy-loading 'services' on Instance uuid 2c0cb519-166b-4f4d-ae3e-46309a6d7c75 {{(pid=117848) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 19 11:12:22.301261 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=1,nova_cell1:INSERT=3 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:12:22.312245 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-66e610ba-f56e-4f64-ac35-507bf719eb60 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "DELETE /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75/os-volume_attachments/854d21ee-4ea2-4b71-8a40-bd0678e1468c" status: 202 len: 0 microversion: 2.1 time: 0.173761 Jul 19 11:12:22.313071 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 599/1190] 10.222.0.33 () {58 vars in 1415 bytes} [Tue Jul 19 11:12:22 2022] DELETE /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75/os-volume_attachments/854d21ee-4ea2-4b71-8a40-bd0678e1468c => generated 0 bytes in 176 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 19 11:12:23.477142 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-c9fbf760-fee3-430e-a7b9-54a279973579 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:23.478149 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-c9fbf760-fee3-430e-a7b9-54a279973579 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:23.484605 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c9fbf760-fee3-430e-a7b9-54a279973579 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:23.485106 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c9fbf760-fee3-430e-a7b9-54a279973579 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:23.485866 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c9fbf760-fee3-430e-a7b9-54a279973579 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:23.486930 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c9fbf760-fee3-430e-a7b9-54a279973579 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:23.515179 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-c9fbf760-fee3-430e-a7b9-54a279973579 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.041826 Jul 19 11:12:23.515664 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 592/1191] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:12:23 2022] GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75/os-volume_attachments => generated 25 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 19 11:12:23.620689 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-2ee137cc-53a8-44a6-b1f8-808c9a711ce8 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:23.621810 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-2ee137cc-53a8-44a6-b1f8-808c9a711ce8 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:23.628613 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2ee137cc-53a8-44a6-b1f8-808c9a711ce8 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:23.629087 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2ee137cc-53a8-44a6-b1f8-808c9a711ce8 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:23.629844 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2ee137cc-53a8-44a6-b1f8-808c9a711ce8 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:23.630279 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2ee137cc-53a8-44a6-b1f8-808c9a711ce8 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:23.659968 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-2ee137cc-53a8-44a6-b1f8-808c9a711ce8 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.044229 Jul 19 11:12:23.660520 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 600/1192] 10.222.0.33 () {58 vars in 1301 bytes} [Tue Jul 19 11:12:23 2022] GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75/os-volume_attachments => generated 25 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 19 11:12:23.717185 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-1a73ba83-1c44-479f-8052-e1c636acc36e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:23.717677 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-1a73ba83-1c44-479f-8052-e1c636acc36e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:23.724187 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1a73ba83-1c44-479f-8052-e1c636acc36e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:23.724633 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1a73ba83-1c44-479f-8052-e1c636acc36e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:23.725415 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1a73ba83-1c44-479f-8052-e1c636acc36e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:23.725848 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1a73ba83-1c44-479f-8052-e1c636acc36e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:23.749577 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-1a73ba83-1c44-479f-8052-e1c636acc36e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Going to try to terminate instance {{(pid=117849) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 19 11:12:23.821490 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-1a73ba83-1c44-479f-8052-e1c636acc36e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "DELETE /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75" status: 204 len: 0 microversion: 2.1 time: 0.107795 Jul 19 11:12:23.821994 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 593/1193] 10.222.0.33 () {58 vars in 1238 bytes} [Tue Jul 19 11:12:23 2022] DELETE /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 19 11:12:23.831233 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-17f434f6-2431-4cb1-935e-098f9666a13b tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:23.831835 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-17f434f6-2431-4cb1-935e-098f9666a13b tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:23.837812 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-17f434f6-2431-4cb1-935e-098f9666a13b tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:23.838254 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-17f434f6-2431-4cb1-935e-098f9666a13b tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:23.839002 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-17f434f6-2431-4cb1-935e-098f9666a13b tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:23.839425 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-17f434f6-2431-4cb1-935e-098f9666a13b tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:23.971503 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-17f434f6-2431-4cb1-935e-098f9666a13b tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:12:23.979520 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-17f434f6-2431-4cb1-935e-098f9666a13b tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75" status: 200 len: 1675 microversion: 2.1 time: 0.153219 Jul 19 11:12:23.980130 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 601/1194] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:12:23 2022] GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75 => generated 1675 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:12:24.998036 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-ae26e6b3-5c83-4ecd-b62a-7406017ba37e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:24.998724 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-ae26e6b3-5c83-4ecd-b62a-7406017ba37e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:25.005902 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ae26e6b3-5c83-4ecd-b62a-7406017ba37e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:25.006349 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ae26e6b3-5c83-4ecd-b62a-7406017ba37e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:25.007133 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ae26e6b3-5c83-4ecd-b62a-7406017ba37e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:25.007561 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ae26e6b3-5c83-4ecd-b62a-7406017ba37e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:25.135041 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-ae26e6b3-5c83-4ecd-b62a-7406017ba37e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:12:25.142492 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-ae26e6b3-5c83-4ecd-b62a-7406017ba37e tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75" status: 200 len: 1675 microversion: 2.1 time: 0.148704 Jul 19 11:12:25.143059 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 594/1195] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:12:24 2022] GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75 => generated 1675 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:12:26.161270 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-3bf545a0-8dfe-4d6a-ac0d-09b32fa7721a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:26.170681 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-3bf545a0-8dfe-4d6a-ac0d-09b32fa7721a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:26.185850 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3bf545a0-8dfe-4d6a-ac0d-09b32fa7721a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:26.186222 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3bf545a0-8dfe-4d6a-ac0d-09b32fa7721a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:26.187026 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3bf545a0-8dfe-4d6a-ac0d-09b32fa7721a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:26.187458 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-3bf545a0-8dfe-4d6a-ac0d-09b32fa7721a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:26.462681 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-3bf545a0-8dfe-4d6a-ac0d-09b32fa7721a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c0b6f4cfede44b2aa8f4a8dfc1e211d6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b5133f102c4606ac7981542949b178', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:12:26.472838 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-3bf545a0-8dfe-4d6a-ac0d-09b32fa7721a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75" status: 200 len: 1675 microversion: 2.1 time: 0.315669 Jul 19 11:12:26.473731 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 602/1196] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:12:26 2022] GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75 => generated 1675 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:12:26.892075 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-25256330-50d8-470b-aafb-7b943de10140 req-dbc06ac2-e874-4d2b-96e5-b526a2c298c9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2c0cb519-166b-4f4d-ae3e-46309a6d7c75", "name": "network-vif-deleted", "tag": "ce50b8aa-9d22-4ed1-84f5-50bee2bf7a72"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:12:26.903005 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-25256330-50d8-470b-aafb-7b943de10140 req-dbc06ac2-e874-4d2b-96e5-b526a2c298c9 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:26.903893 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-25256330-50d8-470b-aafb-7b943de10140 req-dbc06ac2-e874-4d2b-96e5-b526a2c298c9 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:26.924816 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-25256330-50d8-470b-aafb-7b943de10140 req-dbc06ac2-e874-4d2b-96e5-b526a2c298c9 service nova] Creating event network-vif-deleted:ce50b8aa-9d22-4ed1-84f5-50bee2bf7a72 for instance 2c0cb519-166b-4f4d-ae3e-46309a6d7c75 on hv-cinder-maste Jul 19 11:12:26.931182 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-25256330-50d8-470b-aafb-7b943de10140 req-dbc06ac2-e874-4d2b-96e5-b526a2c298c9 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043710 Jul 19 11:12:26.932165 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 595/1197] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:12:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:12:27.492198 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-fded757a-e236-4924-9296-cea314949d0a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:27.492835 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-fded757a-e236-4924-9296-cea314949d0a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:27.499222 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fded757a-e236-4924-9296-cea314949d0a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:27.500828 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fded757a-e236-4924-9296-cea314949d0a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:27.502460 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fded757a-e236-4924-9296-cea314949d0a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:27.503272 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-fded757a-e236-4924-9296-cea314949d0a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:27.520116 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.wsgi [None req-fded757a-e236-4924-9296-cea314949d0a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] HTTP exception thrown: Instance 2c0cb519-166b-4f4d-ae3e-46309a6d7c75 could not be found. Jul 19 11:12:27.522245 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-fded757a-e236-4924-9296-cea314949d0a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Request method failure captured: Jul 19 11:12:27.522245 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: request: GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75 HTTP/1.1 Jul 19 11:12:27.522245 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept: application/json Jul 19 11:12:27.522245 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept-Encoding: identity Jul 19 11:12:27.522245 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Connection: close Jul 19 11:12:27.522245 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Length: 0 Jul 19 11:12:27.522245 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Type: application/json Jul 19 11:12:27.522245 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Host: 10.222.0.33 Jul 19 11:12:27.522245 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Openstack-System-Scope: None Jul 19 11:12:27.522245 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: User-Agent: python-urllib3/1.26.9 Jul 19 11:12:27.522245 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Auth-Token: gAAAAABi1pAu99_KxkXaZby1psxJ1ftONI49OqTcD4XzMi6hyP2ZiznBv9HGO6U0N-i4E5yVLAsyrvg4F0nQOhLM8vQUGcVey3N6L-tk-5qYOymaTM2Q0YoDwJ2T5cyt9gxJB_YaQKNDCdDnlwJK6XuChZuA6B6r4ILZAuFVdn5hixAgJPbdxDY Jul 19 11:12:27.522245 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Id: None Jul 19 11:12:27.522245 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Name: None Jul 19 11:12:27.522245 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Identity-Status: Confirmed Jul 19 11:12:27.522245 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Is-Admin-Project: True Jul 19 11:12:27.522245 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Id: default Jul 19 11:12:27.522245 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Name: Default Jul 19 11:12:27.522245 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Id: 54b5133f102c4606ac7981542949b178 Jul 19 11:12:27.522245 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Name: tempest-AttachVolumeTestJSON-815902756 Jul 19 11:12:27.522245 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Role: member,reader Jul 19 11:12:27.522245 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Roles: member,reader Jul 19 11:12:27.523724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/54b5133f102c4606ac7981542949b178"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/54b5133f102c4606ac7981542949b178"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/54b5133f102c4606ac7981542949b178"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_54b5133f102c4606ac7981542949b178"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:12:27.523724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant: tempest-AttachVolumeTestJSON-815902756 Jul 19 11:12:27.523724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Id: 54b5133f102c4606ac7981542949b178 Jul 19 11:12:27.523724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Name: tempest-AttachVolumeTestJSON-815902756 Jul 19 11:12:27.523724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User: tempest-AttachVolumeTestJSON-815902756-project Jul 19 11:12:27.523724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Id: default Jul 19 11:12:27.523724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Name: Default Jul 19 11:12:27.523724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Id: c0b6f4cfede44b2aa8f4a8dfc1e211d6 Jul 19 11:12:27.523724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Name: tempest-AttachVolumeTestJSON-815902756-project Jul 19 11:12:27.523724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: method: > Jul 19 11:12:27.523724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: exception: Instance 2c0cb519-166b-4f4d-ae3e-46309a6d7c75 could not be found. Jul 19 11:12:27.523724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: action_args: {'id': '2c0cb519-166b-4f4d-ae3e-46309a6d7c75'} Jul 19 11:12:27.524606 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:12:27.524606 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:27.524606 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:12:27.524606 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:12:27.524606 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:12:27.524606 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:12:27.524606 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:12:27.524606 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:12:27.524606 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:12:27.524606 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:12:27.524606 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2c0cb519-166b-4f4d-ae3e-46309a6d7c75 could not be found. Jul 19 11:12:27.524606 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:27.524606 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:27.524606 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:27.524606 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:27.524606 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:27.524606 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:27.524606 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:12:27.524606 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:12:27.524606 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:12:27.525932 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:12:27.525932 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 19 11:12:27.525932 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 19 11:12:27.525932 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:12:27.525932 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:12:27.525932 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:12:27.525932 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:12:27.525932 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2c0cb519-166b-4f4d-ae3e-46309a6d7c75 could not be found. Jul 19 11:12:27.525932 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:27.525932 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:27.525932 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:27.525932 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:27.525932 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:27.525932 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:27.525932 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:12:27.525932 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:12:27.525932 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2c0cb519-166b-4f4d-ae3e-46309a6d7c75 could not be found. Jul 19 11:12:27.525932 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:27.528526 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-fded757a-e236-4924-9296-cea314949d0a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Returning 404 to user: Instance 2c0cb519-166b-4f4d-ae3e-46309a6d7c75 could not be found. {{(pid=117848) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:12:27.529665 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-fded757a-e236-4924-9296-cea314949d0a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75" status: 404 len: 111 microversion: 2.1 time: 0.041564 Jul 19 11:12:27.530600 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 603/1198] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:12:27 2022] GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:12:27.540201 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-4f4bada4-b78a-41b1-bcc7-09c3805b768f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '.version_select at 0x7f04aa149550>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:27.569506 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-4f4bada4-b78a-41b1-bcc7-09c3805b768f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1910311538" status: 202 len: 0 microversion: 2.1 time: 0.033540 Jul 19 11:12:27.570125 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 596/1199] 10.222.0.33 () {58 vars in 1220 bytes} [Tue Jul 19 11:12:27 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1910311538 => generated 0 bytes in 35 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 19 11:12:30.636427 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-9ab5bdb8-7d44-4417-9746-7d0de7371e17 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:30.637055 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-9ab5bdb8-7d44-4417-9746-7d0de7371e17 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:30.643888 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9ab5bdb8-7d44-4417-9746-7d0de7371e17 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:30.644327 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9ab5bdb8-7d44-4417-9746-7d0de7371e17 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:30.645117 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9ab5bdb8-7d44-4417-9746-7d0de7371e17 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:30.645544 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9ab5bdb8-7d44-4417-9746-7d0de7371e17 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:30.659065 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.wsgi [None req-9ab5bdb8-7d44-4417-9746-7d0de7371e17 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] HTTP exception thrown: Instance 2c0cb519-166b-4f4d-ae3e-46309a6d7c75 could not be found. Jul 19 11:12:30.660865 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-9ab5bdb8-7d44-4417-9746-7d0de7371e17 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Request method failure captured: Jul 19 11:12:30.660865 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: request: DELETE /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75 HTTP/1.1 Jul 19 11:12:30.660865 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept: application/json Jul 19 11:12:30.660865 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept-Encoding: identity Jul 19 11:12:30.660865 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Connection: close Jul 19 11:12:30.660865 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Length: 0 Jul 19 11:12:30.660865 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Type: application/json Jul 19 11:12:30.660865 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Host: 10.222.0.33 Jul 19 11:12:30.660865 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Openstack-System-Scope: None Jul 19 11:12:30.660865 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: User-Agent: python-urllib3/1.26.9 Jul 19 11:12:30.660865 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Auth-Token: gAAAAABi1pAu99_KxkXaZby1psxJ1ftONI49OqTcD4XzMi6hyP2ZiznBv9HGO6U0N-i4E5yVLAsyrvg4F0nQOhLM8vQUGcVey3N6L-tk-5qYOymaTM2Q0YoDwJ2T5cyt9gxJB_YaQKNDCdDnlwJK6XuChZuA6B6r4ILZAuFVdn5hixAgJPbdxDY Jul 19 11:12:30.660865 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Id: None Jul 19 11:12:30.660865 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Name: None Jul 19 11:12:30.660865 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Identity-Status: Confirmed Jul 19 11:12:30.660865 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Is-Admin-Project: True Jul 19 11:12:30.660865 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Id: default Jul 19 11:12:30.660865 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Name: Default Jul 19 11:12:30.660865 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Id: 54b5133f102c4606ac7981542949b178 Jul 19 11:12:30.660865 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Name: tempest-AttachVolumeTestJSON-815902756 Jul 19 11:12:30.660865 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Role: member,reader Jul 19 11:12:30.660865 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Roles: member,reader Jul 19 11:12:30.662406 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/54b5133f102c4606ac7981542949b178"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/54b5133f102c4606ac7981542949b178"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/54b5133f102c4606ac7981542949b178"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_54b5133f102c4606ac7981542949b178"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:12:30.662406 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant: tempest-AttachVolumeTestJSON-815902756 Jul 19 11:12:30.662406 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Id: 54b5133f102c4606ac7981542949b178 Jul 19 11:12:30.662406 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Name: tempest-AttachVolumeTestJSON-815902756 Jul 19 11:12:30.662406 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User: tempest-AttachVolumeTestJSON-815902756-project Jul 19 11:12:30.662406 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Id: default Jul 19 11:12:30.662406 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Name: Default Jul 19 11:12:30.662406 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Id: c0b6f4cfede44b2aa8f4a8dfc1e211d6 Jul 19 11:12:30.662406 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Name: tempest-AttachVolumeTestJSON-815902756-project Jul 19 11:12:30.662406 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: method: > Jul 19 11:12:30.662406 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: exception: Instance 2c0cb519-166b-4f4d-ae3e-46309a6d7c75 could not be found. Jul 19 11:12:30.662406 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: action_args: {'id': '2c0cb519-166b-4f4d-ae3e-46309a6d7c75'} Jul 19 11:12:30.665995 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:12:30.665995 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:30.665995 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:12:30.665995 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:12:30.665995 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:12:30.665995 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:12:30.665995 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:12:30.665995 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:12:30.665995 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:12:30.665995 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:12:30.665995 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2c0cb519-166b-4f4d-ae3e-46309a6d7c75 could not be found. Jul 19 11:12:30.665995 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:30.665995 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:30.665995 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:30.665995 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:30.665995 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:30.665995 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:30.665995 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:12:30.665995 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:12:30.665995 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:12:30.667437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:12:30.667437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 19 11:12:30.667437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 19 11:12:30.667437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 19 11:12:30.667437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 19 11:12:30.667437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:12:30.667437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:12:30.667437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:12:30.667437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:12:30.667437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2c0cb519-166b-4f4d-ae3e-46309a6d7c75 could not be found. Jul 19 11:12:30.667437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:30.667437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:30.667437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:30.667437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:30.667437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:30.667437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:30.667437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:12:30.667437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:12:30.667437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2c0cb519-166b-4f4d-ae3e-46309a6d7c75 could not be found. Jul 19 11:12:30.667437 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:30.668884 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-9ab5bdb8-7d44-4417-9746-7d0de7371e17 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Returning 404 to user: Instance 2c0cb519-166b-4f4d-ae3e-46309a6d7c75 could not be found. {{(pid=117848) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:12:30.668884 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-9ab5bdb8-7d44-4417-9746-7d0de7371e17 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "DELETE /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75" status: 404 len: 111 microversion: 2.1 time: 0.029763 Jul 19 11:12:30.668884 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 604/1200] 10.222.0.33 () {58 vars in 1238 bytes} [Tue Jul 19 11:12:30 2022] DELETE /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:12:30.671098 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-6101897c-71b3-49ae-a7a1-8f982981f02a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:30.671854 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-6101897c-71b3-49ae-a7a1-8f982981f02a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 2c0cb519-166b-4f4d-ae3e-46309a6d7c75] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:30.678021 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6101897c-71b3-49ae-a7a1-8f982981f02a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:30.678468 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6101897c-71b3-49ae-a7a1-8f982981f02a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:30.679222 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6101897c-71b3-49ae-a7a1-8f982981f02a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:30.679658 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6101897c-71b3-49ae-a7a1-8f982981f02a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:30.693132 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.wsgi [None req-6101897c-71b3-49ae-a7a1-8f982981f02a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] HTTP exception thrown: Instance 2c0cb519-166b-4f4d-ae3e-46309a6d7c75 could not be found. Jul 19 11:12:30.694735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-6101897c-71b3-49ae-a7a1-8f982981f02a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Request method failure captured: Jul 19 11:12:30.694735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: request: GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75 HTTP/1.1 Jul 19 11:12:30.694735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept: application/json Jul 19 11:12:30.694735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept-Encoding: identity Jul 19 11:12:30.694735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Connection: close Jul 19 11:12:30.694735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Length: 0 Jul 19 11:12:30.694735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Type: application/json Jul 19 11:12:30.694735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Host: 10.222.0.33 Jul 19 11:12:30.694735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Openstack-System-Scope: None Jul 19 11:12:30.694735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: User-Agent: python-urllib3/1.26.9 Jul 19 11:12:30.694735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Auth-Token: gAAAAABi1pAu99_KxkXaZby1psxJ1ftONI49OqTcD4XzMi6hyP2ZiznBv9HGO6U0N-i4E5yVLAsyrvg4F0nQOhLM8vQUGcVey3N6L-tk-5qYOymaTM2Q0YoDwJ2T5cyt9gxJB_YaQKNDCdDnlwJK6XuChZuA6B6r4ILZAuFVdn5hixAgJPbdxDY Jul 19 11:12:30.694735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Id: None Jul 19 11:12:30.694735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Name: None Jul 19 11:12:30.694735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Identity-Status: Confirmed Jul 19 11:12:30.694735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Is-Admin-Project: True Jul 19 11:12:30.694735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Id: default Jul 19 11:12:30.694735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Name: Default Jul 19 11:12:30.694735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Id: 54b5133f102c4606ac7981542949b178 Jul 19 11:12:30.694735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Name: tempest-AttachVolumeTestJSON-815902756 Jul 19 11:12:30.694735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Role: member,reader Jul 19 11:12:30.694735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Roles: member,reader Jul 19 11:12:30.696233 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/54b5133f102c4606ac7981542949b178"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/54b5133f102c4606ac7981542949b178"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/54b5133f102c4606ac7981542949b178"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_54b5133f102c4606ac7981542949b178"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:12:30.696233 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant: tempest-AttachVolumeTestJSON-815902756 Jul 19 11:12:30.696233 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Id: 54b5133f102c4606ac7981542949b178 Jul 19 11:12:30.696233 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Name: tempest-AttachVolumeTestJSON-815902756 Jul 19 11:12:30.696233 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User: tempest-AttachVolumeTestJSON-815902756-project Jul 19 11:12:30.696233 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Id: default Jul 19 11:12:30.696233 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Name: Default Jul 19 11:12:30.696233 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Id: c0b6f4cfede44b2aa8f4a8dfc1e211d6 Jul 19 11:12:30.696233 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Name: tempest-AttachVolumeTestJSON-815902756-project Jul 19 11:12:30.696233 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: method: > Jul 19 11:12:30.696233 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: exception: Instance 2c0cb519-166b-4f4d-ae3e-46309a6d7c75 could not be found. Jul 19 11:12:30.696233 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: action_args: {'id': '2c0cb519-166b-4f4d-ae3e-46309a6d7c75'} Jul 19 11:12:30.697140 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:12:30.697140 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:30.697140 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:12:30.697140 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:12:30.697140 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:12:30.697140 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:12:30.697140 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:12:30.697140 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:12:30.697140 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:12:30.697140 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:12:30.697140 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2c0cb519-166b-4f4d-ae3e-46309a6d7c75 could not be found. Jul 19 11:12:30.697140 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:30.697140 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:30.697140 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:30.697140 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:30.697140 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:30.697140 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:30.697140 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:12:30.697140 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:12:30.697140 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:12:30.698633 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:12:30.698633 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 19 11:12:30.698633 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 19 11:12:30.698633 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:12:30.698633 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:12:30.698633 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:12:30.698633 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:12:30.698633 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2c0cb519-166b-4f4d-ae3e-46309a6d7c75 could not be found. Jul 19 11:12:30.698633 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:30.698633 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:30.698633 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:30.698633 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:30.698633 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:30.698633 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:30.698633 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:12:30.698633 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:12:30.698633 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2c0cb519-166b-4f4d-ae3e-46309a6d7c75 could not be found. Jul 19 11:12:30.698633 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:30.699900 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-6101897c-71b3-49ae-a7a1-8f982981f02a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Returning 404 to user: Instance 2c0cb519-166b-4f4d-ae3e-46309a6d7c75 could not be found. {{(pid=117849) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:12:30.699900 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-6101897c-71b3-49ae-a7a1-8f982981f02a tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75" status: 404 len: 111 microversion: 2.1 time: 0.029503 Jul 19 11:12:30.699900 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 597/1201] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:12:30 2022] GET /compute/v2.1/servers/2c0cb519-166b-4f4d-ae3e-46309a6d7c75 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 19 11:12:31.832218 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-1c6118cf-0d2e-47c4-83d5-faa3a4921638 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:31.833350 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-1c6118cf-0d2e-47c4-83d5-faa3a4921638 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:31.839652 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1c6118cf-0d2e-47c4-83d5-faa3a4921638 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:31.840232 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1c6118cf-0d2e-47c4-83d5-faa3a4921638 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:31.841137 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1c6118cf-0d2e-47c4-83d5-faa3a4921638 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:31.841655 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1c6118cf-0d2e-47c4-83d5-faa3a4921638 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:31.855722 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.wsgi [None req-1c6118cf-0d2e-47c4-83d5-faa3a4921638 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] HTTP exception thrown: Instance 940e8a5c-b365-40c2-8361-fbb677969487 could not be found. Jul 19 11:12:31.857368 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-1c6118cf-0d2e-47c4-83d5-faa3a4921638 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Request method failure captured: Jul 19 11:12:31.857368 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: request: DELETE /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 HTTP/1.1 Jul 19 11:12:31.857368 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept: application/json Jul 19 11:12:31.857368 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept-Encoding: identity Jul 19 11:12:31.857368 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Connection: close Jul 19 11:12:31.857368 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Length: 0 Jul 19 11:12:31.857368 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Type: application/json Jul 19 11:12:31.857368 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Host: 10.222.0.33 Jul 19 11:12:31.857368 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Openstack-System-Scope: None Jul 19 11:12:31.857368 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: User-Agent: python-urllib3/1.26.9 Jul 19 11:12:31.857368 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Auth-Token: gAAAAABi1pAu99_KxkXaZby1psxJ1ftONI49OqTcD4XzMi6hyP2ZiznBv9HGO6U0N-i4E5yVLAsyrvg4F0nQOhLM8vQUGcVey3N6L-tk-5qYOymaTM2Q0YoDwJ2T5cyt9gxJB_YaQKNDCdDnlwJK6XuChZuA6B6r4ILZAuFVdn5hixAgJPbdxDY Jul 19 11:12:31.857368 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Id: None Jul 19 11:12:31.857368 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Name: None Jul 19 11:12:31.857368 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Identity-Status: Confirmed Jul 19 11:12:31.857368 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Is-Admin-Project: True Jul 19 11:12:31.857368 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Id: default Jul 19 11:12:31.857368 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Name: Default Jul 19 11:12:31.857368 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Id: 54b5133f102c4606ac7981542949b178 Jul 19 11:12:31.857368 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Name: tempest-AttachVolumeTestJSON-815902756 Jul 19 11:12:31.857368 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Role: member,reader Jul 19 11:12:31.857368 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Roles: member,reader Jul 19 11:12:31.859000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/54b5133f102c4606ac7981542949b178"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/54b5133f102c4606ac7981542949b178"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/54b5133f102c4606ac7981542949b178"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_54b5133f102c4606ac7981542949b178"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:12:31.859000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant: tempest-AttachVolumeTestJSON-815902756 Jul 19 11:12:31.859000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Id: 54b5133f102c4606ac7981542949b178 Jul 19 11:12:31.859000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Name: tempest-AttachVolumeTestJSON-815902756 Jul 19 11:12:31.859000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User: tempest-AttachVolumeTestJSON-815902756-project Jul 19 11:12:31.859000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Id: default Jul 19 11:12:31.859000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Name: Default Jul 19 11:12:31.859000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Id: c0b6f4cfede44b2aa8f4a8dfc1e211d6 Jul 19 11:12:31.859000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Name: tempest-AttachVolumeTestJSON-815902756-project Jul 19 11:12:31.859000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: method: > Jul 19 11:12:31.859000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: exception: Instance 940e8a5c-b365-40c2-8361-fbb677969487 could not be found. Jul 19 11:12:31.859000 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: action_args: {'id': '940e8a5c-b365-40c2-8361-fbb677969487'} Jul 19 11:12:31.860515 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:12:31.860515 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:31.860515 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:12:31.860515 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:12:31.860515 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:12:31.860515 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:12:31.860515 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:12:31.860515 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:12:31.860515 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:12:31.860515 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:12:31.860515 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 940e8a5c-b365-40c2-8361-fbb677969487 could not be found. Jul 19 11:12:31.860515 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:31.860515 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:31.860515 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:31.860515 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:31.860515 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:31.860515 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:31.860515 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:12:31.860515 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:12:31.860515 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:12:31.862068 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:12:31.862068 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 19 11:12:31.862068 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 19 11:12:31.862068 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 19 11:12:31.862068 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 19 11:12:31.862068 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:12:31.862068 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:12:31.862068 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:12:31.862068 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:12:31.862068 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 940e8a5c-b365-40c2-8361-fbb677969487 could not be found. Jul 19 11:12:31.862068 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:31.862068 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:31.862068 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:31.862068 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:31.862068 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:31.862068 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:31.862068 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:12:31.862068 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:12:31.862068 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 940e8a5c-b365-40c2-8361-fbb677969487 could not be found. Jul 19 11:12:31.862068 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:31.864625 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-1c6118cf-0d2e-47c4-83d5-faa3a4921638 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Returning 404 to user: Instance 940e8a5c-b365-40c2-8361-fbb677969487 could not be found. {{(pid=117848) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:12:31.864625 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-1c6118cf-0d2e-47c4-83d5-faa3a4921638 tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "DELETE /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 404 len: 111 microversion: 2.1 time: 0.032435 Jul 19 11:12:31.864625 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 605/1202] 10.222.0.33 () {58 vars in 1238 bytes} [Tue Jul 19 11:12:31 2022] DELETE /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:12:31.869298 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-57c5115b-0cf1-43a3-ae08-9c93d674df0f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:31.870209 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-57c5115b-0cf1-43a3-ae08-9c93d674df0f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] [instance: 940e8a5c-b365-40c2-8361-fbb677969487] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:31.876616 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-57c5115b-0cf1-43a3-ae08-9c93d674df0f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:31.877224 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-57c5115b-0cf1-43a3-ae08-9c93d674df0f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:31.878037 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-57c5115b-0cf1-43a3-ae08-9c93d674df0f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:31.878643 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-57c5115b-0cf1-43a3-ae08-9c93d674df0f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:31.893260 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.wsgi [None req-57c5115b-0cf1-43a3-ae08-9c93d674df0f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] HTTP exception thrown: Instance 940e8a5c-b365-40c2-8361-fbb677969487 could not be found. Jul 19 11:12:31.894889 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-57c5115b-0cf1-43a3-ae08-9c93d674df0f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Request method failure captured: Jul 19 11:12:31.894889 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: request: GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 HTTP/1.1 Jul 19 11:12:31.894889 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept: application/json Jul 19 11:12:31.894889 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept-Encoding: identity Jul 19 11:12:31.894889 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Connection: close Jul 19 11:12:31.894889 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Length: 0 Jul 19 11:12:31.894889 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Type: application/json Jul 19 11:12:31.894889 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Host: 10.222.0.33 Jul 19 11:12:31.894889 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Openstack-System-Scope: None Jul 19 11:12:31.894889 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: User-Agent: python-urllib3/1.26.9 Jul 19 11:12:31.894889 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Auth-Token: gAAAAABi1pAu99_KxkXaZby1psxJ1ftONI49OqTcD4XzMi6hyP2ZiznBv9HGO6U0N-i4E5yVLAsyrvg4F0nQOhLM8vQUGcVey3N6L-tk-5qYOymaTM2Q0YoDwJ2T5cyt9gxJB_YaQKNDCdDnlwJK6XuChZuA6B6r4ILZAuFVdn5hixAgJPbdxDY Jul 19 11:12:31.894889 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Id: None Jul 19 11:12:31.894889 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Name: None Jul 19 11:12:31.894889 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Identity-Status: Confirmed Jul 19 11:12:31.894889 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Is-Admin-Project: True Jul 19 11:12:31.894889 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Id: default Jul 19 11:12:31.894889 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Name: Default Jul 19 11:12:31.894889 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Id: 54b5133f102c4606ac7981542949b178 Jul 19 11:12:31.894889 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Name: tempest-AttachVolumeTestJSON-815902756 Jul 19 11:12:31.894889 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Role: member,reader Jul 19 11:12:31.894889 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Roles: member,reader Jul 19 11:12:31.896330 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/54b5133f102c4606ac7981542949b178"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/54b5133f102c4606ac7981542949b178"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/54b5133f102c4606ac7981542949b178"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_54b5133f102c4606ac7981542949b178"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:12:31.896330 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant: tempest-AttachVolumeTestJSON-815902756 Jul 19 11:12:31.896330 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Id: 54b5133f102c4606ac7981542949b178 Jul 19 11:12:31.896330 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Name: tempest-AttachVolumeTestJSON-815902756 Jul 19 11:12:31.896330 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User: tempest-AttachVolumeTestJSON-815902756-project Jul 19 11:12:31.896330 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Id: default Jul 19 11:12:31.896330 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Name: Default Jul 19 11:12:31.896330 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Id: c0b6f4cfede44b2aa8f4a8dfc1e211d6 Jul 19 11:12:31.896330 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Name: tempest-AttachVolumeTestJSON-815902756-project Jul 19 11:12:31.896330 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: method: > Jul 19 11:12:31.896330 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: exception: Instance 940e8a5c-b365-40c2-8361-fbb677969487 could not be found. Jul 19 11:12:31.896330 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: action_args: {'id': '940e8a5c-b365-40c2-8361-fbb677969487'} Jul 19 11:12:31.897363 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:12:31.897363 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:31.897363 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:12:31.897363 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:12:31.897363 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:12:31.897363 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:12:31.897363 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:12:31.897363 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:12:31.897363 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:12:31.897363 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:12:31.897363 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 940e8a5c-b365-40c2-8361-fbb677969487 could not be found. Jul 19 11:12:31.897363 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:31.897363 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:31.897363 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:31.897363 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:31.897363 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:31.897363 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:31.897363 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:12:31.897363 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:12:31.897363 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:12:31.898836 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:12:31.898836 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 19 11:12:31.898836 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 19 11:12:31.898836 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:12:31.898836 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:12:31.898836 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:12:31.898836 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:12:31.898836 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 940e8a5c-b365-40c2-8361-fbb677969487 could not be found. Jul 19 11:12:31.898836 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:31.898836 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:31.898836 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:31.898836 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:31.898836 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:31.898836 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:31.898836 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:12:31.898836 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:12:31.898836 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 940e8a5c-b365-40c2-8361-fbb677969487 could not be found. Jul 19 11:12:31.898836 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:31.900120 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-57c5115b-0cf1-43a3-ae08-9c93d674df0f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] Returning 404 to user: Instance 940e8a5c-b365-40c2-8361-fbb677969487 could not be found. {{(pid=117849) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:12:31.900120 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-57c5115b-0cf1-43a3-ae08-9c93d674df0f tempest-AttachVolumeTestJSON-815902756 tempest-AttachVolumeTestJSON-815902756-project] 10.222.0.33 "GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487" status: 404 len: 111 microversion: 2.1 time: 0.031347 Jul 19 11:12:31.900120 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 598/1203] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:12:31 2022] GET /compute/v2.1/servers/940e8a5c-b365-40c2-8361-fbb677969487 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 19 11:12:37.085294 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-9be8a3b3-403b-49ba-be2d-0152cff03c80 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:37.086204 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-9be8a3b3-403b-49ba-be2d-0152cff03c80 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:37.088669 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_api:SELECT=35,nova_api:DELETE=1 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:12:37.093943 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9be8a3b3-403b-49ba-be2d-0152cff03c80 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:37.094384 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9be8a3b3-403b-49ba-be2d-0152cff03c80 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:37.095170 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9be8a3b3-403b-49ba-be2d-0152cff03c80 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:37.095673 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-9be8a3b3-403b-49ba-be2d-0152cff03c80 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:37.126188 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-9be8a3b3-403b-49ba-be2d-0152cff03c80 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.044806 Jul 19 11:12:37.127346 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 606/1204] 10.222.0.33 () {60 vars in 1342 bytes} [Tue Jul 19 11:12:37 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9/os-volume_attachments => generated 25 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:12:37.185610 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-4c832c22-68d0-4801-adbe-26a77e790c1c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:37.186087 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-4c832c22-68d0-4801-adbe-26a77e790c1c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:37.194083 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4c832c22-68d0-4801-adbe-26a77e790c1c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:37.194528 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4c832c22-68d0-4801-adbe-26a77e790c1c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:37.195473 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4c832c22-68d0-4801-adbe-26a77e790c1c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:37.196106 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4c832c22-68d0-4801-adbe-26a77e790c1c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:37.221884 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-4c832c22-68d0-4801-adbe-26a77e790c1c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Going to try to terminate instance {{(pid=117849) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 19 11:12:37.329505 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-4c832c22-68d0-4801-adbe-26a77e790c1c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "DELETE /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 204 len: 0 microversion: 2.20 time: 0.147174 Jul 19 11:12:37.333987 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 599/1205] 10.222.0.33 () {60 vars in 1279 bytes} [Tue Jul 19 11:12:37 2022] DELETE /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 19 11:12:37.339769 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-59c8dcea-3cfa-47a5-a1eb-cbbe7f8a11d6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:37.340460 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-59c8dcea-3cfa-47a5-a1eb-cbbe7f8a11d6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:37.349983 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-59c8dcea-3cfa-47a5-a1eb-cbbe7f8a11d6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:37.350523 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-59c8dcea-3cfa-47a5-a1eb-cbbe7f8a11d6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:37.351696 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-59c8dcea-3cfa-47a5-a1eb-cbbe7f8a11d6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:37.352271 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-59c8dcea-3cfa-47a5-a1eb-cbbe7f8a11d6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:37.527014 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-59c8dcea-3cfa-47a5-a1eb-cbbe7f8a11d6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:12:37.539950 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-59c8dcea-3cfa-47a5-a1eb-cbbe7f8a11d6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:12:37.541853 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-59c8dcea-3cfa-47a5-a1eb-cbbe7f8a11d6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:12:37.543307 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-59c8dcea-3cfa-47a5-a1eb-cbbe7f8a11d6 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1723 microversion: 2.20 time: 0.208098 Jul 19 11:12:37.544023 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 607/1206] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:12:37 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1723 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 19 11:12:38.562996 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-3456b8f7-f5ea-4e13-a471-cb2f0731160c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:38.564912 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-3456b8f7-f5ea-4e13-a471-cb2f0731160c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:38.573123 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3456b8f7-f5ea-4e13-a471-cb2f0731160c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:38.573850 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3456b8f7-f5ea-4e13-a471-cb2f0731160c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:38.574725 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3456b8f7-f5ea-4e13-a471-cb2f0731160c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:38.575418 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3456b8f7-f5ea-4e13-a471-cb2f0731160c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:38.806085 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-3456b8f7-f5ea-4e13-a471-cb2f0731160c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:12:38.815107 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-3456b8f7-f5ea-4e13-a471-cb2f0731160c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:12:38.816945 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-3456b8f7-f5ea-4e13-a471-cb2f0731160c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:12:38.818528 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-3456b8f7-f5ea-4e13-a471-cb2f0731160c tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1723 microversion: 2.20 time: 0.259713 Jul 19 11:12:38.819080 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 600/1207] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:12:38 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1723 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:12:39.839684 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-a748ac33-b58e-4080-a55a-5af64d2e6d24 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:39.840518 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-a748ac33-b58e-4080-a55a-5af64d2e6d24 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:39.850572 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a748ac33-b58e-4080-a55a-5af64d2e6d24 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:39.851048 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a748ac33-b58e-4080-a55a-5af64d2e6d24 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:39.851905 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a748ac33-b58e-4080-a55a-5af64d2e6d24 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:39.852289 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-a748ac33-b58e-4080-a55a-5af64d2e6d24 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:39.873896 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-85de90e3-31b2-495d-b1a0-598d627ee26d req-35213112-b0f1-45a6-95fb-f745dec06d52 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "36abf2ae-3b4c-4095-b9ba-2bd738e266d9", "name": "network-vif-deleted", "tag": "049ef62d-5f12-4134-9010-2bc20edecd41"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:12:39.902425 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-85de90e3-31b2-495d-b1a0-598d627ee26d req-35213112-b0f1-45a6-95fb-f745dec06d52 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:39.906980 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-85de90e3-31b2-495d-b1a0-598d627ee26d req-35213112-b0f1-45a6-95fb-f745dec06d52 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:39.957351 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-85de90e3-31b2-495d-b1a0-598d627ee26d req-35213112-b0f1-45a6-95fb-f745dec06d52 service nova] Creating event network-vif-deleted:049ef62d-5f12-4134-9010-2bc20edecd41 for instance 36abf2ae-3b4c-4095-b9ba-2bd738e266d9 on hv-cinder-maste Jul 19 11:12:39.962759 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-85de90e3-31b2-495d-b1a0-598d627ee26d req-35213112-b0f1-45a6-95fb-f745dec06d52 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.092649 Jul 19 11:12:39.963952 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 601/1208] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:12:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:12:40.026527 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-a748ac33-b58e-4080-a55a-5af64d2e6d24 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:12:40.050536 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-a748ac33-b58e-4080-a55a-5af64d2e6d24 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:12:40.060954 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-a748ac33-b58e-4080-a55a-5af64d2e6d24 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '246dc188d9a44b1cb38721e3dd8a863a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e1d59596ecae4e288733b90c577aa86b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:12:40.064556 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-a748ac33-b58e-4080-a55a-5af64d2e6d24 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 200 len: 1655 microversion: 2.20 time: 0.229433 Jul 19 11:12:40.079364 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 608/1209] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:12:39 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 1655 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 19 11:12:41.089239 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-a565faf4-4d3c-4e5f-99e6-9e200a6baa42 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:41.090610 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-a565faf4-4d3c-4e5f-99e6-9e200a6baa42 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:41.093508 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_api:SELECT=39,nova_api:UPDATE=2,nova_api:DELETE=1 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:12:41.102071 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a565faf4-4d3c-4e5f-99e6-9e200a6baa42 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:41.102748 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a565faf4-4d3c-4e5f-99e6-9e200a6baa42 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:41.104597 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a565faf4-4d3c-4e5f-99e6-9e200a6baa42 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:41.105223 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a565faf4-4d3c-4e5f-99e6-9e200a6baa42 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:41.119297 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=83,nova_cell1:UPDATE=4,nova_cell1:INSERT=3 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:12:41.122132 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.wsgi [None req-a565faf4-4d3c-4e5f-99e6-9e200a6baa42 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] HTTP exception thrown: Instance 36abf2ae-3b4c-4095-b9ba-2bd738e266d9 could not be found. Jul 19 11:12:41.124063 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-a565faf4-4d3c-4e5f-99e6-9e200a6baa42 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Request method failure captured: Jul 19 11:12:41.124063 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: request: GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 HTTP/1.1 Jul 19 11:12:41.124063 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept: application/json Jul 19 11:12:41.124063 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept-Encoding: identity Jul 19 11:12:41.124063 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Connection: close Jul 19 11:12:41.124063 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Length: 0 Jul 19 11:12:41.124063 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Type: application/json Jul 19 11:12:41.124063 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Host: 10.222.0.33 Jul 19 11:12:41.124063 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Openstack-System-Scope: None Jul 19 11:12:41.124063 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: User-Agent: python-urllib3/1.26.9 Jul 19 11:12:41.124063 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Auth-Token: gAAAAABi1o_H80dr4M-lIIT6uzG8exhA4LKN984_YaeFvAcM9fuolWzoNXAcL7qhFNcYKUqYeIgZl9w0RVHbKPy5b2-j3djdpYY7wzh5yonBpziIeFPtCUxYkrQU736pgDwXX6Psqz45jhFgkLySjLfsmzUJa9qWn_nUPS3R9xp-POD7aFDj-oE Jul 19 11:12:41.124063 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Id: None Jul 19 11:12:41.124063 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Name: None Jul 19 11:12:41.124063 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Identity-Status: Confirmed Jul 19 11:12:41.124063 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Is-Admin-Project: True Jul 19 11:12:41.124063 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Openstack-Nova-Api-Version: 2.20 Jul 19 11:12:41.124063 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Id: default Jul 19 11:12:41.124063 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Name: Default Jul 19 11:12:41.124063 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Id: e1d59596ecae4e288733b90c577aa86b Jul 19 11:12:41.124063 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-36195510 Jul 19 11:12:41.124063 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Role: member,reader Jul 19 11:12:41.124063 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Roles: member,reader Jul 19 11:12:41.125992 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/e1d59596ecae4e288733b90c577aa86b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/e1d59596ecae4e288733b90c577aa86b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/e1d59596ecae4e288733b90c577aa86b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_e1d59596ecae4e288733b90c577aa86b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:12:41.125992 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant: tempest-AttachVolumeShelveTestJSON-36195510 Jul 19 11:12:41.125992 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Id: e1d59596ecae4e288733b90c577aa86b Jul 19 11:12:41.125992 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-36195510 Jul 19 11:12:41.125992 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User: tempest-AttachVolumeShelveTestJSON-36195510-project Jul 19 11:12:41.125992 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Id: default Jul 19 11:12:41.125992 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Name: Default Jul 19 11:12:41.125992 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Id: 246dc188d9a44b1cb38721e3dd8a863a Jul 19 11:12:41.125992 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Name: tempest-AttachVolumeShelveTestJSON-36195510-project Jul 19 11:12:41.125992 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: method: > Jul 19 11:12:41.125992 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: exception: Instance 36abf2ae-3b4c-4095-b9ba-2bd738e266d9 could not be found. Jul 19 11:12:41.125992 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: action_args: {'id': '36abf2ae-3b4c-4095-b9ba-2bd738e266d9'} Jul 19 11:12:41.126782 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:12:41.126782 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:41.126782 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:12:41.126782 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:12:41.126782 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:12:41.126782 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:12:41.126782 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:12:41.126782 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:12:41.126782 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:12:41.126782 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:12:41.126782 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 36abf2ae-3b4c-4095-b9ba-2bd738e266d9 could not be found. Jul 19 11:12:41.126782 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:41.126782 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:41.126782 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:41.126782 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:41.126782 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:41.126782 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:41.126782 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:12:41.126782 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:12:41.126782 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:12:41.128025 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:12:41.128025 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 19 11:12:41.128025 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 19 11:12:41.128025 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:12:41.128025 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:12:41.128025 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:12:41.128025 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:12:41.128025 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 36abf2ae-3b4c-4095-b9ba-2bd738e266d9 could not be found. Jul 19 11:12:41.128025 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:41.128025 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:41.128025 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:41.128025 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:41.128025 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:41.128025 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:41.128025 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:12:41.128025 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:12:41.128025 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 36abf2ae-3b4c-4095-b9ba-2bd738e266d9 could not be found. Jul 19 11:12:41.128025 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:41.130342 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-a565faf4-4d3c-4e5f-99e6-9e200a6baa42 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Returning 404 to user: Instance 36abf2ae-3b4c-4095-b9ba-2bd738e266d9 could not be found. {{(pid=117849) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:12:41.131196 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-a565faf4-4d3c-4e5f-99e6-9e200a6baa42 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 404 len: 111 microversion: 2.20 time: 0.048528 Jul 19 11:12:41.135975 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 602/1210] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:12:41 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 111 bytes in 54 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 19 11:12:41.142653 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-e32016bb-dd80-46a3-b38c-224422a2b985 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '.version_select at 0x7f04aa5f3430>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:41.173633 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-e32016bb-dd80-46a3-b38c-224422a2b985 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1639386872" status: 204 len: 0 microversion: 2.20 time: 0.036613 Jul 19 11:12:41.174709 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 609/1211] 10.222.0.33 () {60 vars in 1261 bytes} [Tue Jul 19 11:12:41 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1639386872 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 19 11:12:43.091315 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-7aca5b4b-88cd-4a50-850a-4f35071d89b8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:43.091935 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-7aca5b4b-88cd-4a50-850a-4f35071d89b8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:43.099819 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7aca5b4b-88cd-4a50-850a-4f35071d89b8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:43.100356 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7aca5b4b-88cd-4a50-850a-4f35071d89b8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:43.101151 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7aca5b4b-88cd-4a50-850a-4f35071d89b8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:43.101590 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-7aca5b4b-88cd-4a50-850a-4f35071d89b8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:43.117064 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.wsgi [None req-7aca5b4b-88cd-4a50-850a-4f35071d89b8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] HTTP exception thrown: Instance 36abf2ae-3b4c-4095-b9ba-2bd738e266d9 could not be found. Jul 19 11:12:43.118620 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-7aca5b4b-88cd-4a50-850a-4f35071d89b8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Request method failure captured: Jul 19 11:12:43.118620 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: request: DELETE /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 HTTP/1.1 Jul 19 11:12:43.118620 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept: application/json Jul 19 11:12:43.118620 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept-Encoding: identity Jul 19 11:12:43.118620 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Connection: close Jul 19 11:12:43.118620 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Length: 0 Jul 19 11:12:43.118620 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Type: application/json Jul 19 11:12:43.118620 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Host: 10.222.0.33 Jul 19 11:12:43.118620 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Openstack-System-Scope: None Jul 19 11:12:43.118620 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: User-Agent: python-urllib3/1.26.9 Jul 19 11:12:43.118620 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Auth-Token: gAAAAABi1o_H80dr4M-lIIT6uzG8exhA4LKN984_YaeFvAcM9fuolWzoNXAcL7qhFNcYKUqYeIgZl9w0RVHbKPy5b2-j3djdpYY7wzh5yonBpziIeFPtCUxYkrQU736pgDwXX6Psqz45jhFgkLySjLfsmzUJa9qWn_nUPS3R9xp-POD7aFDj-oE Jul 19 11:12:43.118620 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Id: None Jul 19 11:12:43.118620 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Name: None Jul 19 11:12:43.118620 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Identity-Status: Confirmed Jul 19 11:12:43.118620 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Is-Admin-Project: True Jul 19 11:12:43.118620 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Openstack-Nova-Api-Version: 2.20 Jul 19 11:12:43.118620 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Id: default Jul 19 11:12:43.118620 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Name: Default Jul 19 11:12:43.118620 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Id: e1d59596ecae4e288733b90c577aa86b Jul 19 11:12:43.118620 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-36195510 Jul 19 11:12:43.118620 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Role: member,reader Jul 19 11:12:43.118620 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Roles: member,reader Jul 19 11:12:43.120225 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/e1d59596ecae4e288733b90c577aa86b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/e1d59596ecae4e288733b90c577aa86b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/e1d59596ecae4e288733b90c577aa86b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_e1d59596ecae4e288733b90c577aa86b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:12:43.120225 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant: tempest-AttachVolumeShelveTestJSON-36195510 Jul 19 11:12:43.120225 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Id: e1d59596ecae4e288733b90c577aa86b Jul 19 11:12:43.120225 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-36195510 Jul 19 11:12:43.120225 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User: tempest-AttachVolumeShelveTestJSON-36195510-project Jul 19 11:12:43.120225 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Id: default Jul 19 11:12:43.120225 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Name: Default Jul 19 11:12:43.120225 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Id: 246dc188d9a44b1cb38721e3dd8a863a Jul 19 11:12:43.120225 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Name: tempest-AttachVolumeShelveTestJSON-36195510-project Jul 19 11:12:43.120225 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: method: > Jul 19 11:12:43.120225 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: exception: Instance 36abf2ae-3b4c-4095-b9ba-2bd738e266d9 could not be found. Jul 19 11:12:43.120225 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: action_args: {'id': '36abf2ae-3b4c-4095-b9ba-2bd738e266d9'} Jul 19 11:12:43.121016 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:12:43.121016 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:43.121016 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:12:43.121016 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:12:43.121016 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:12:43.121016 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:12:43.121016 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:12:43.121016 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:12:43.121016 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:12:43.121016 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:12:43.121016 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 36abf2ae-3b4c-4095-b9ba-2bd738e266d9 could not be found. Jul 19 11:12:43.121016 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:43.121016 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:43.121016 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:43.121016 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:43.121016 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:43.121016 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:43.121016 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:12:43.121016 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:12:43.121016 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:12:43.122553 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:12:43.122553 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 19 11:12:43.122553 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 19 11:12:43.122553 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 19 11:12:43.122553 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 19 11:12:43.122553 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:12:43.122553 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:12:43.122553 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:12:43.122553 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:12:43.122553 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 36abf2ae-3b4c-4095-b9ba-2bd738e266d9 could not be found. Jul 19 11:12:43.122553 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:43.122553 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:43.122553 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:43.122553 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:43.122553 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:43.122553 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:43.122553 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:12:43.122553 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:12:43.122553 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 36abf2ae-3b4c-4095-b9ba-2bd738e266d9 could not be found. Jul 19 11:12:43.122553 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:43.124962 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-7aca5b4b-88cd-4a50-850a-4f35071d89b8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Returning 404 to user: Instance 36abf2ae-3b4c-4095-b9ba-2bd738e266d9 could not be found. {{(pid=117849) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:12:43.124962 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-7aca5b4b-88cd-4a50-850a-4f35071d89b8 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "DELETE /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 404 len: 111 microversion: 2.20 time: 0.032590 Jul 19 11:12:43.130509 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-97e8085e-c486-4fb0-97d7-21f5e7c90e12 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:43.131320 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-97e8085e-c486-4fb0-97d7-21f5e7c90e12 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: 36abf2ae-3b4c-4095-b9ba-2bd738e266d9] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:43.139656 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-97e8085e-c486-4fb0-97d7-21f5e7c90e12 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:43.140207 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-97e8085e-c486-4fb0-97d7-21f5e7c90e12 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:43.140860 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 603/1212] 10.222.0.33 () {60 vars in 1279 bytes} [Tue Jul 19 11:12:43 2022] DELETE /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 111 bytes in 54 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 19 11:12:43.141171 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-97e8085e-c486-4fb0-97d7-21f5e7c90e12 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:43.141774 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-97e8085e-c486-4fb0-97d7-21f5e7c90e12 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:43.159575 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.wsgi [None req-97e8085e-c486-4fb0-97d7-21f5e7c90e12 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] HTTP exception thrown: Instance 36abf2ae-3b4c-4095-b9ba-2bd738e266d9 could not be found. Jul 19 11:12:43.161153 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-97e8085e-c486-4fb0-97d7-21f5e7c90e12 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Request method failure captured: Jul 19 11:12:43.161153 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: request: GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 HTTP/1.1 Jul 19 11:12:43.161153 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept: application/json Jul 19 11:12:43.161153 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept-Encoding: identity Jul 19 11:12:43.161153 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Connection: close Jul 19 11:12:43.161153 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Length: 0 Jul 19 11:12:43.161153 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Type: application/json Jul 19 11:12:43.161153 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Host: 10.222.0.33 Jul 19 11:12:43.161153 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Openstack-System-Scope: None Jul 19 11:12:43.161153 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: User-Agent: python-urllib3/1.26.9 Jul 19 11:12:43.161153 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Auth-Token: gAAAAABi1o_H80dr4M-lIIT6uzG8exhA4LKN984_YaeFvAcM9fuolWzoNXAcL7qhFNcYKUqYeIgZl9w0RVHbKPy5b2-j3djdpYY7wzh5yonBpziIeFPtCUxYkrQU736pgDwXX6Psqz45jhFgkLySjLfsmzUJa9qWn_nUPS3R9xp-POD7aFDj-oE Jul 19 11:12:43.161153 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Id: None Jul 19 11:12:43.161153 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Name: None Jul 19 11:12:43.161153 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Identity-Status: Confirmed Jul 19 11:12:43.161153 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Is-Admin-Project: True Jul 19 11:12:43.161153 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Openstack-Nova-Api-Version: 2.20 Jul 19 11:12:43.161153 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Id: default Jul 19 11:12:43.161153 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Name: Default Jul 19 11:12:43.161153 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Id: e1d59596ecae4e288733b90c577aa86b Jul 19 11:12:43.161153 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-36195510 Jul 19 11:12:43.161153 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Role: member,reader Jul 19 11:12:43.161153 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Roles: member,reader Jul 19 11:12:43.162639 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/e1d59596ecae4e288733b90c577aa86b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/e1d59596ecae4e288733b90c577aa86b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/e1d59596ecae4e288733b90c577aa86b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_e1d59596ecae4e288733b90c577aa86b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:12:43.162639 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant: tempest-AttachVolumeShelveTestJSON-36195510 Jul 19 11:12:43.162639 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Id: e1d59596ecae4e288733b90c577aa86b Jul 19 11:12:43.162639 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-36195510 Jul 19 11:12:43.162639 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User: tempest-AttachVolumeShelveTestJSON-36195510-project Jul 19 11:12:43.162639 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Id: default Jul 19 11:12:43.162639 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Name: Default Jul 19 11:12:43.162639 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Id: 246dc188d9a44b1cb38721e3dd8a863a Jul 19 11:12:43.162639 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Name: tempest-AttachVolumeShelveTestJSON-36195510-project Jul 19 11:12:43.162639 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: method: > Jul 19 11:12:43.162639 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: exception: Instance 36abf2ae-3b4c-4095-b9ba-2bd738e266d9 could not be found. Jul 19 11:12:43.162639 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: action_args: {'id': '36abf2ae-3b4c-4095-b9ba-2bd738e266d9'} Jul 19 11:12:43.164208 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:12:43.164208 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:43.164208 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:12:43.164208 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:12:43.164208 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:12:43.164208 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:12:43.164208 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:12:43.164208 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:12:43.164208 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:12:43.164208 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:12:43.164208 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 36abf2ae-3b4c-4095-b9ba-2bd738e266d9 could not be found. Jul 19 11:12:43.164208 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:43.164208 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:43.164208 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:43.164208 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:43.164208 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:43.164208 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:43.164208 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:12:43.164208 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:12:43.164208 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:12:43.165528 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:12:43.165528 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 19 11:12:43.165528 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 19 11:12:43.165528 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:12:43.165528 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:12:43.165528 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:12:43.165528 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:12:43.165528 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 36abf2ae-3b4c-4095-b9ba-2bd738e266d9 could not be found. Jul 19 11:12:43.165528 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:43.165528 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:43.165528 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:43.165528 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:43.165528 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:43.165528 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:43.165528 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:12:43.165528 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:12:43.165528 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 36abf2ae-3b4c-4095-b9ba-2bd738e266d9 could not be found. Jul 19 11:12:43.165528 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:43.167210 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-97e8085e-c486-4fb0-97d7-21f5e7c90e12 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Returning 404 to user: Instance 36abf2ae-3b4c-4095-b9ba-2bd738e266d9 could not be found. {{(pid=117848) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:12:43.167210 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-97e8085e-c486-4fb0-97d7-21f5e7c90e12 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9" status: 404 len: 111 microversion: 2.20 time: 0.037275 Jul 19 11:12:43.167210 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 610/1213] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:12:43 2022] GET /compute/v2.1/servers/36abf2ae-3b4c-4095-b9ba-2bd738e266d9 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Jul 19 11:12:44.304049 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-1e8fda94-bc19-478b-ade4-c40ecfd39939 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:44.304509 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-1e8fda94-bc19-478b-ade4-c40ecfd39939 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:44.310871 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1e8fda94-bc19-478b-ade4-c40ecfd39939 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:44.311323 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1e8fda94-bc19-478b-ade4-c40ecfd39939 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:44.312117 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1e8fda94-bc19-478b-ade4-c40ecfd39939 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:44.312556 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1e8fda94-bc19-478b-ade4-c40ecfd39939 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:44.326820 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.wsgi [None req-1e8fda94-bc19-478b-ade4-c40ecfd39939 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] HTTP exception thrown: Instance f52fbfae-577b-403d-a4d3-9a4b79ae2326 could not be found. Jul 19 11:12:44.328370 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-1e8fda94-bc19-478b-ade4-c40ecfd39939 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Request method failure captured: Jul 19 11:12:44.328370 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: request: DELETE /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 HTTP/1.1 Jul 19 11:12:44.328370 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept: application/json Jul 19 11:12:44.328370 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept-Encoding: identity Jul 19 11:12:44.328370 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Connection: close Jul 19 11:12:44.328370 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Length: 0 Jul 19 11:12:44.328370 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Type: application/json Jul 19 11:12:44.328370 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Host: 10.222.0.33 Jul 19 11:12:44.328370 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Openstack-System-Scope: None Jul 19 11:12:44.328370 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: User-Agent: python-urllib3/1.26.9 Jul 19 11:12:44.328370 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Auth-Token: gAAAAABi1o_H80dr4M-lIIT6uzG8exhA4LKN984_YaeFvAcM9fuolWzoNXAcL7qhFNcYKUqYeIgZl9w0RVHbKPy5b2-j3djdpYY7wzh5yonBpziIeFPtCUxYkrQU736pgDwXX6Psqz45jhFgkLySjLfsmzUJa9qWn_nUPS3R9xp-POD7aFDj-oE Jul 19 11:12:44.328370 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Id: None Jul 19 11:12:44.328370 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Name: None Jul 19 11:12:44.328370 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Identity-Status: Confirmed Jul 19 11:12:44.328370 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Is-Admin-Project: True Jul 19 11:12:44.328370 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Openstack-Nova-Api-Version: 2.20 Jul 19 11:12:44.328370 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Id: default Jul 19 11:12:44.328370 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Name: Default Jul 19 11:12:44.328370 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Id: e1d59596ecae4e288733b90c577aa86b Jul 19 11:12:44.328370 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-36195510 Jul 19 11:12:44.328370 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Role: member,reader Jul 19 11:12:44.328370 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Roles: member,reader Jul 19 11:12:44.330464 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/e1d59596ecae4e288733b90c577aa86b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/e1d59596ecae4e288733b90c577aa86b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/e1d59596ecae4e288733b90c577aa86b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_e1d59596ecae4e288733b90c577aa86b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:12:44.330464 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant: tempest-AttachVolumeShelveTestJSON-36195510 Jul 19 11:12:44.330464 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Id: e1d59596ecae4e288733b90c577aa86b Jul 19 11:12:44.330464 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-36195510 Jul 19 11:12:44.330464 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User: tempest-AttachVolumeShelveTestJSON-36195510-project Jul 19 11:12:44.330464 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Id: default Jul 19 11:12:44.330464 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Name: Default Jul 19 11:12:44.330464 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Id: 246dc188d9a44b1cb38721e3dd8a863a Jul 19 11:12:44.330464 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Name: tempest-AttachVolumeShelveTestJSON-36195510-project Jul 19 11:12:44.330464 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: method: > Jul 19 11:12:44.330464 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: exception: Instance f52fbfae-577b-403d-a4d3-9a4b79ae2326 could not be found. Jul 19 11:12:44.330464 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: action_args: {'id': 'f52fbfae-577b-403d-a4d3-9a4b79ae2326'} Jul 19 11:12:44.331444 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:12:44.331444 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:44.331444 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:12:44.331444 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:12:44.331444 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:12:44.331444 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:12:44.331444 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:12:44.331444 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:12:44.331444 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:12:44.331444 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:12:44.331444 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f52fbfae-577b-403d-a4d3-9a4b79ae2326 could not be found. Jul 19 11:12:44.331444 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:44.331444 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:44.331444 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:44.331444 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:44.331444 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:44.331444 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:44.331444 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:12:44.331444 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:12:44.331444 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:12:44.332643 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:12:44.332643 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 19 11:12:44.332643 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 19 11:12:44.332643 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 19 11:12:44.332643 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 19 11:12:44.332643 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:12:44.332643 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:12:44.332643 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:12:44.332643 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:12:44.332643 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f52fbfae-577b-403d-a4d3-9a4b79ae2326 could not be found. Jul 19 11:12:44.332643 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:44.332643 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:44.332643 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:44.332643 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:44.332643 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:44.332643 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:44.332643 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:12:44.332643 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:12:44.332643 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f52fbfae-577b-403d-a4d3-9a4b79ae2326 could not be found. Jul 19 11:12:44.332643 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:12:44.335177 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-1e8fda94-bc19-478b-ade4-c40ecfd39939 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Returning 404 to user: Instance f52fbfae-577b-403d-a4d3-9a4b79ae2326 could not be found. {{(pid=117849) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:12:44.335177 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-1e8fda94-bc19-478b-ade4-c40ecfd39939 tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "DELETE /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 404 len: 111 microversion: 2.20 time: 0.030837 Jul 19 11:12:44.335177 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 604/1214] 10.222.0.33 () {60 vars in 1279 bytes} [Tue Jul 19 11:12:44 2022] DELETE /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 19 11:12:44.338562 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-88cf2004-c147-475d-a1f7-e2743934691e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:12:44.339166 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-88cf2004-c147-475d-a1f7-e2743934691e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] [instance: f52fbfae-577b-403d-a4d3-9a4b79ae2326] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:12:44.345443 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-88cf2004-c147-475d-a1f7-e2743934691e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:44.345896 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-88cf2004-c147-475d-a1f7-e2743934691e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:44.346670 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-88cf2004-c147-475d-a1f7-e2743934691e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:12:44.347090 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-88cf2004-c147-475d-a1f7-e2743934691e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:12:44.360803 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.wsgi [None req-88cf2004-c147-475d-a1f7-e2743934691e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] HTTP exception thrown: Instance f52fbfae-577b-403d-a4d3-9a4b79ae2326 could not be found. Jul 19 11:12:44.362331 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-88cf2004-c147-475d-a1f7-e2743934691e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Request method failure captured: Jul 19 11:12:44.362331 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: request: GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 HTTP/1.1 Jul 19 11:12:44.362331 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept: application/json Jul 19 11:12:44.362331 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept-Encoding: identity Jul 19 11:12:44.362331 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Connection: close Jul 19 11:12:44.362331 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Length: 0 Jul 19 11:12:44.362331 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Type: application/json Jul 19 11:12:44.362331 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Host: 10.222.0.33 Jul 19 11:12:44.362331 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Openstack-System-Scope: None Jul 19 11:12:44.362331 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: User-Agent: python-urllib3/1.26.9 Jul 19 11:12:44.362331 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Auth-Token: gAAAAABi1o_H80dr4M-lIIT6uzG8exhA4LKN984_YaeFvAcM9fuolWzoNXAcL7qhFNcYKUqYeIgZl9w0RVHbKPy5b2-j3djdpYY7wzh5yonBpziIeFPtCUxYkrQU736pgDwXX6Psqz45jhFgkLySjLfsmzUJa9qWn_nUPS3R9xp-POD7aFDj-oE Jul 19 11:12:44.362331 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Id: None Jul 19 11:12:44.362331 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Name: None Jul 19 11:12:44.362331 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Identity-Status: Confirmed Jul 19 11:12:44.362331 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Is-Admin-Project: True Jul 19 11:12:44.362331 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Openstack-Nova-Api-Version: 2.20 Jul 19 11:12:44.362331 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Id: default Jul 19 11:12:44.362331 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Name: Default Jul 19 11:12:44.362331 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Id: e1d59596ecae4e288733b90c577aa86b Jul 19 11:12:44.362331 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-36195510 Jul 19 11:12:44.362331 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Role: member,reader Jul 19 11:12:44.362331 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Roles: member,reader Jul 19 11:12:44.363867 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/e1d59596ecae4e288733b90c577aa86b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/e1d59596ecae4e288733b90c577aa86b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/e1d59596ecae4e288733b90c577aa86b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_e1d59596ecae4e288733b90c577aa86b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:12:44.363867 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant: tempest-AttachVolumeShelveTestJSON-36195510 Jul 19 11:12:44.363867 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Id: e1d59596ecae4e288733b90c577aa86b Jul 19 11:12:44.363867 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-36195510 Jul 19 11:12:44.363867 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User: tempest-AttachVolumeShelveTestJSON-36195510-project Jul 19 11:12:44.363867 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Id: default Jul 19 11:12:44.363867 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Name: Default Jul 19 11:12:44.363867 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Id: 246dc188d9a44b1cb38721e3dd8a863a Jul 19 11:12:44.363867 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Name: tempest-AttachVolumeShelveTestJSON-36195510-project Jul 19 11:12:44.363867 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: method: > Jul 19 11:12:44.363867 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: exception: Instance f52fbfae-577b-403d-a4d3-9a4b79ae2326 could not be found. Jul 19 11:12:44.363867 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: action_args: {'id': 'f52fbfae-577b-403d-a4d3-9a4b79ae2326'} Jul 19 11:12:44.365216 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:12:44.365216 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:44.365216 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:12:44.365216 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:12:44.365216 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:12:44.365216 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:12:44.365216 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:12:44.365216 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:12:44.365216 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:12:44.365216 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:12:44.365216 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f52fbfae-577b-403d-a4d3-9a4b79ae2326 could not be found. Jul 19 11:12:44.365216 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:44.365216 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:44.365216 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:44.365216 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:44.365216 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:44.365216 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:44.365216 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:12:44.365216 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:12:44.365216 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:12:44.366476 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:12:44.366476 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 19 11:12:44.366476 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 19 11:12:44.366476 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:12:44.366476 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:12:44.366476 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:12:44.366476 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:12:44.366476 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f52fbfae-577b-403d-a4d3-9a4b79ae2326 could not be found. Jul 19 11:12:44.366476 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:44.366476 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:12:44.366476 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:44.366476 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:12:44.366476 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:12:44.366476 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:12:44.366476 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:12:44.366476 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:12:44.366476 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f52fbfae-577b-403d-a4d3-9a4b79ae2326 could not be found. Jul 19 11:12:44.366476 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:12:44.368004 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-88cf2004-c147-475d-a1f7-e2743934691e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] Returning 404 to user: Instance f52fbfae-577b-403d-a4d3-9a4b79ae2326 could not be found. {{(pid=117848) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:12:44.368004 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-88cf2004-c147-475d-a1f7-e2743934691e tempest-AttachVolumeShelveTestJSON-36195510 tempest-AttachVolumeShelveTestJSON-36195510-project] 10.222.0.33 "GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326" status: 404 len: 111 microversion: 2.20 time: 0.028883 Jul 19 11:12:44.368004 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 611/1215] 10.222.0.33 () {60 vars in 1276 bytes} [Tue Jul 19 11:12:44 2022] GET /compute/v2.1/servers/f52fbfae-577b-403d-a4d3-9a4b79ae2326 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 19 11:13:43.969775 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-6657b897-65b3-42d1-9b2f-c2df74537030 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-888760943", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1205270003", "security_groups": [{"name": "tempest-secgroup-smoke-1334719817"}], "block_device_mapping_v2": [{"uuid": "588c03df-e4dc-4d5e-8028-f30f94cf4380", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "1660664f-e111-4b9d-93da-a8c613b3e4df"}]}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:13:43.979408 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_api:SELECT=5 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:13:43.980059 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=5 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:13:44.089934 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.network.neutron [None req-6657b897-65b3-42d1-9b2f-c2df74537030 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] validate_networks() for [('1660664f-e111-4b9d-93da-a8c613b3e4df', None, None, None, None, None)] {{(pid=117849) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 19 11:13:44.468617 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-6657b897-65b3-42d1-9b2f-c2df74537030 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Flavor limits 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 19 11:13:44.469075 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-6657b897-65b3-42d1-9b2f-c2df74537030 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Image limits 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 19 11:13:44.469555 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-6657b897-65b3-42d1-9b2f-c2df74537030 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Flavor pref 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 19 11:13:44.469963 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-6657b897-65b3-42d1-9b2f-c2df74537030 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Image pref 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 19 11:13:44.470372 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-6657b897-65b3-42d1-9b2f-c2df74537030 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 19 11:13:44.479239 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6657b897-65b3-42d1-9b2f-c2df74537030 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:13:44.479699 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6657b897-65b3-42d1-9b2f-c2df74537030 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:13:44.480495 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6657b897-65b3-42d1-9b2f-c2df74537030 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:13:44.480960 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6657b897-65b3-42d1-9b2f-c2df74537030 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:13:44.501269 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-6657b897-65b3-42d1-9b2f-c2df74537030 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Getting quotas for project a7109adf88ac4391aee35c150e96e0b1. Resources: {'cores', 'instances', 'ram'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:13:44.507951 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-6657b897-65b3-42d1-9b2f-c2df74537030 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Getting quotas for user a217e983effa43f3a4f84221984189c5 and project a7109adf88ac4391aee35c150e96e0b1. Resources: {'cores', 'instances', 'ram'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:13:44.567199 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-6657b897-65b3-42d1-9b2f-c2df74537030 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Going to run 1 instances... {{(pid=117849) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 19 11:13:44.569306 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-6657b897-65b3-42d1-9b2f-c2df74537030 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: f3c3d260-c061-48b7-8233-a3c3ef00d062] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='588c03df-e4dc-4d5e-8028-f30f94cf4380',volume_size=None,volume_type=None)] {{(pid=117849) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 19 11:13:44.698308 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-6657b897-65b3-42d1-9b2f-c2df74537030 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.733261 Jul 19 11:13:44.704034 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 605/1216] 10.222.0.33 () {60 vars in 1146 bytes} [Tue Jul 19 11:13:43 2022] POST /compute/v2.1/servers => generated 406 bytes in 735 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 19 11:13:44.715993 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-3aa6eeb7-bd0a-4ebe-8a3c-523081515937 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:13:44.716622 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-3aa6eeb7-bd0a-4ebe-8a3c-523081515937 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: f3c3d260-c061-48b7-8233-a3c3ef00d062] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:13:44.718753 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_api:SELECT=12,nova_api:DELETE=1 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:13:44.719447 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=46 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:13:44.786212 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-3aa6eeb7-bd0a-4ebe-8a3c-523081515937 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:13:44.793333 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-3aa6eeb7-bd0a-4ebe-8a3c-523081515937 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062" status: 200 len: 1094 microversion: 2.1 time: 0.088404 Jul 19 11:13:44.795050 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 612/1217] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:13:44 2022] GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062 => generated 1094 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:13:45.813134 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-5d03b4a9-2cd1-4082-86c6-ae2d098bf3b2 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:13:45.814456 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-5d03b4a9-2cd1-4082-86c6-ae2d098bf3b2 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: f3c3d260-c061-48b7-8233-a3c3ef00d062] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:13:45.835094 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5d03b4a9-2cd1-4082-86c6-ae2d098bf3b2 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:13:45.836644 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5d03b4a9-2cd1-4082-86c6-ae2d098bf3b2 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:13:45.839537 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5d03b4a9-2cd1-4082-86c6-ae2d098bf3b2 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:13:45.840016 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5d03b4a9-2cd1-4082-86c6-ae2d098bf3b2 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:13:45.949660 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-5d03b4a9-2cd1-4082-86c6-ae2d098bf3b2 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:13:45.958044 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-5d03b4a9-2cd1-4082-86c6-ae2d098bf3b2 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062" status: 200 len: 1210 microversion: 2.1 time: 0.151421 Jul 19 11:13:45.959246 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 606/1218] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:13:45 2022] GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062 => generated 1210 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:13:46.977610 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-7b3c49af-90d2-4d3d-ab34-eaa18edf2bd5 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:13:46.978745 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-7b3c49af-90d2-4d3d-ab34-eaa18edf2bd5 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: f3c3d260-c061-48b7-8233-a3c3ef00d062] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:13:46.989785 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7b3c49af-90d2-4d3d-ab34-eaa18edf2bd5 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:13:46.990661 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7b3c49af-90d2-4d3d-ab34-eaa18edf2bd5 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:13:46.991704 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7b3c49af-90d2-4d3d-ab34-eaa18edf2bd5 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:13:46.992217 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-7b3c49af-90d2-4d3d-ab34-eaa18edf2bd5 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:13:47.205434 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-7b3c49af-90d2-4d3d-ab34-eaa18edf2bd5 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:13:47.214720 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-7b3c49af-90d2-4d3d-ab34-eaa18edf2bd5 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062" status: 200 len: 1266 microversion: 2.1 time: 0.242564 Jul 19 11:13:47.224991 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 613/1219] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:13:46 2022] GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062 => generated 1266 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:13:47.503193 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-f2fbc861-966e-4796-9de8-1e69ae4cd583 req-7bbbc178-6cf0-40e8-be0c-17729cf9d13b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f3c3d260-c061-48b7-8233-a3c3ef00d062", "tag": "0518c0aa-dba9-44fd-93b2-3630f121afee"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:13:47.512832 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-f2fbc861-966e-4796-9de8-1e69ae4cd583 req-7bbbc178-6cf0-40e8-be0c-17729cf9d13b service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:13:47.513313 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-f2fbc861-966e-4796-9de8-1e69ae4cd583 req-7bbbc178-6cf0-40e8-be0c-17729cf9d13b service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:13:47.530520 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-f2fbc861-966e-4796-9de8-1e69ae4cd583 req-7bbbc178-6cf0-40e8-be0c-17729cf9d13b service nova] Creating event network-changed:0518c0aa-dba9-44fd-93b2-3630f121afee for instance f3c3d260-c061-48b7-8233-a3c3ef00d062 on hv-cinder-maste Jul 19 11:13:47.537163 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-f2fbc861-966e-4796-9de8-1e69ae4cd583 req-7bbbc178-6cf0-40e8-be0c-17729cf9d13b service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038694 Jul 19 11:13:47.537866 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 607/1220] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:13:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:13:48.231886 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-767caa4e-ef6a-4eb2-b1cd-0abd4f0dc019 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:13:48.232494 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-767caa4e-ef6a-4eb2-b1cd-0abd4f0dc019 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: f3c3d260-c061-48b7-8233-a3c3ef00d062] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:13:48.239463 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-767caa4e-ef6a-4eb2-b1cd-0abd4f0dc019 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:13:48.239914 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-767caa4e-ef6a-4eb2-b1cd-0abd4f0dc019 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:13:48.240682 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-767caa4e-ef6a-4eb2-b1cd-0abd4f0dc019 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:13:48.241158 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-767caa4e-ef6a-4eb2-b1cd-0abd4f0dc019 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:13:48.429390 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-767caa4e-ef6a-4eb2-b1cd-0abd4f0dc019 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:13:48.437694 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-767caa4e-ef6a-4eb2-b1cd-0abd4f0dc019 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062" status: 200 len: 1266 microversion: 2.1 time: 0.209336 Jul 19 11:13:48.438683 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 614/1221] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:13:48 2022] GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062 => generated 1266 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:13:49.456832 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-3d8c7c91-ffca-44b2-af54-0a3d234dd53e tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:13:49.457522 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-3d8c7c91-ffca-44b2-af54-0a3d234dd53e tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: f3c3d260-c061-48b7-8233-a3c3ef00d062] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:13:49.463830 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3d8c7c91-ffca-44b2-af54-0a3d234dd53e tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:13:49.464275 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3d8c7c91-ffca-44b2-af54-0a3d234dd53e tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:13:49.465052 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3d8c7c91-ffca-44b2-af54-0a3d234dd53e tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:13:49.465486 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-3d8c7c91-ffca-44b2-af54-0a3d234dd53e tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:13:49.594654 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-3d8c7c91-ffca-44b2-af54-0a3d234dd53e tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:13:49.602420 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-3d8c7c91-ffca-44b2-af54-0a3d234dd53e tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062" status: 200 len: 1266 microversion: 2.1 time: 0.150493 Jul 19 11:13:49.602989 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 608/1222] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:13:49 2022] GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062 => generated 1266 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:13:50.618964 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-0eada659-5483-4cf1-b74e-1bd9ac5d8891 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:13:50.619528 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-0eada659-5483-4cf1-b74e-1bd9ac5d8891 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: f3c3d260-c061-48b7-8233-a3c3ef00d062] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:13:50.626257 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0eada659-5483-4cf1-b74e-1bd9ac5d8891 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:13:50.626703 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0eada659-5483-4cf1-b74e-1bd9ac5d8891 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:13:50.627458 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0eada659-5483-4cf1-b74e-1bd9ac5d8891 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:13:50.627888 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0eada659-5483-4cf1-b74e-1bd9ac5d8891 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:13:50.755645 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-0eada659-5483-4cf1-b74e-1bd9ac5d8891 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:13:50.762892 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-0eada659-5483-4cf1-b74e-1bd9ac5d8891 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062" status: 200 len: 1266 microversion: 2.1 time: 0.147850 Jul 19 11:13:50.763452 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 615/1223] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:13:50 2022] GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062 => generated 1266 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:13:51.780446 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-4840b19f-4ba3-4222-b438-d09b8bafa0f9 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:13:51.781105 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-4840b19f-4ba3-4222-b438-d09b8bafa0f9 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: f3c3d260-c061-48b7-8233-a3c3ef00d062] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:13:51.788642 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4840b19f-4ba3-4222-b438-d09b8bafa0f9 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:13:51.789529 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4840b19f-4ba3-4222-b438-d09b8bafa0f9 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:13:51.789955 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4840b19f-4ba3-4222-b438-d09b8bafa0f9 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:13:51.790478 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4840b19f-4ba3-4222-b438-d09b8bafa0f9 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:13:51.951993 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-4840b19f-4ba3-4222-b438-d09b8bafa0f9 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:13:51.960077 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-4840b19f-4ba3-4222-b438-d09b8bafa0f9 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062" status: 200 len: 1266 microversion: 2.1 time: 0.183516 Jul 19 11:13:51.960636 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 609/1224] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:13:51 2022] GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062 => generated 1266 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:13:52.977422 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-0da1b927-1c27-4390-af25-3890008e180d tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:13:52.978101 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-0da1b927-1c27-4390-af25-3890008e180d tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: f3c3d260-c061-48b7-8233-a3c3ef00d062] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:13:52.985331 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0da1b927-1c27-4390-af25-3890008e180d tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:13:52.985765 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0da1b927-1c27-4390-af25-3890008e180d tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:13:52.986666 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0da1b927-1c27-4390-af25-3890008e180d tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:13:52.987592 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0da1b927-1c27-4390-af25-3890008e180d tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:13:53.138505 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-0da1b927-1c27-4390-af25-3890008e180d tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:13:53.147875 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-0da1b927-1c27-4390-af25-3890008e180d tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062" status: 200 len: 1266 microversion: 2.1 time: 0.174407 Jul 19 11:13:53.148470 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 616/1225] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:13:52 2022] GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062 => generated 1266 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:13:54.165082 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-0c8fbdb1-c7c2-4560-aa23-c80a309e0f55 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:13:54.165684 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-0c8fbdb1-c7c2-4560-aa23-c80a309e0f55 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: f3c3d260-c061-48b7-8233-a3c3ef00d062] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:13:54.171937 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0c8fbdb1-c7c2-4560-aa23-c80a309e0f55 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:13:54.172383 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0c8fbdb1-c7c2-4560-aa23-c80a309e0f55 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:13:54.173163 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0c8fbdb1-c7c2-4560-aa23-c80a309e0f55 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:13:54.173588 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0c8fbdb1-c7c2-4560-aa23-c80a309e0f55 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:13:54.306007 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-0c8fbdb1-c7c2-4560-aa23-c80a309e0f55 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:13:54.314005 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-0c8fbdb1-c7c2-4560-aa23-c80a309e0f55 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062" status: 200 len: 1266 microversion: 2.1 time: 0.152587 Jul 19 11:13:54.314654 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 610/1226] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:13:54 2022] GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062 => generated 1266 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:13:55.331850 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-39b3b667-6fb4-4c9f-95e1-f07de1857304 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:13:55.332402 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-39b3b667-6fb4-4c9f-95e1-f07de1857304 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: f3c3d260-c061-48b7-8233-a3c3ef00d062] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:13:55.334799 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell0:SELECT=2 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:13:55.340121 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-39b3b667-6fb4-4c9f-95e1-f07de1857304 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:13:55.340567 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-39b3b667-6fb4-4c9f-95e1-f07de1857304 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:13:55.341362 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-39b3b667-6fb4-4c9f-95e1-f07de1857304 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:13:55.341793 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-39b3b667-6fb4-4c9f-95e1-f07de1857304 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:13:55.491168 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-39b3b667-6fb4-4c9f-95e1-f07de1857304 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:13:55.499122 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-39b3b667-6fb4-4c9f-95e1-f07de1857304 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062" status: 200 len: 1266 microversion: 2.1 time: 0.171701 Jul 19 11:13:55.499703 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 617/1227] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:13:55 2022] GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062 => generated 1266 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:13:56.516638 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-bd76c062-c8f2-4d96-aafa-3890d5a21d07 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:13:56.517293 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-bd76c062-c8f2-4d96-aafa-3890d5a21d07 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: f3c3d260-c061-48b7-8233-a3c3ef00d062] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:13:56.519389 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell0:SELECT=2 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:13:56.520028 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell0:SELECT=6 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:13:56.526744 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bd76c062-c8f2-4d96-aafa-3890d5a21d07 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:13:56.527190 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bd76c062-c8f2-4d96-aafa-3890d5a21d07 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:13:56.527938 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bd76c062-c8f2-4d96-aafa-3890d5a21d07 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:13:56.528369 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bd76c062-c8f2-4d96-aafa-3890d5a21d07 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:13:56.655519 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-bd76c062-c8f2-4d96-aafa-3890d5a21d07 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:13:56.663473 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-bd76c062-c8f2-4d96-aafa-3890d5a21d07 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062" status: 200 len: 1266 microversion: 2.1 time: 0.150730 Jul 19 11:13:56.664771 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 611/1228] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:13:56 2022] GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062 => generated 1266 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:13:57.647453 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-ab1846d6-5d47-401e-966b-90f2a9ef3348 req-e1f92f56-bed8-4ac7-af70-77d8bcbb0444 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f3c3d260-c061-48b7-8233-a3c3ef00d062", "name": "network-vif-plugged", "status": "completed", "tag": "0518c0aa-dba9-44fd-93b2-3630f121afee"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:13:57.656587 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-ab1846d6-5d47-401e-966b-90f2a9ef3348 req-e1f92f56-bed8-4ac7-af70-77d8bcbb0444 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:13:57.657272 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-ab1846d6-5d47-401e-966b-90f2a9ef3348 req-e1f92f56-bed8-4ac7-af70-77d8bcbb0444 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:13:57.671487 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-ab1846d6-5d47-401e-966b-90f2a9ef3348 req-e1f92f56-bed8-4ac7-af70-77d8bcbb0444 service nova] Creating event network-vif-plugged:0518c0aa-dba9-44fd-93b2-3630f121afee for instance f3c3d260-c061-48b7-8233-a3c3ef00d062 on hv-cinder-maste Jul 19 11:13:57.677500 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-ab1846d6-5d47-401e-966b-90f2a9ef3348 req-e1f92f56-bed8-4ac7-af70-77d8bcbb0444 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035399 Jul 19 11:13:57.679118 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 618/1229] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:13:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:13:57.681000 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-cd005184-2e86-413d-b220-727e25cc2062 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:13:57.681606 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-cd005184-2e86-413d-b220-727e25cc2062 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: f3c3d260-c061-48b7-8233-a3c3ef00d062] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:13:57.687865 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cd005184-2e86-413d-b220-727e25cc2062 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:13:57.688310 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cd005184-2e86-413d-b220-727e25cc2062 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:13:57.689102 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cd005184-2e86-413d-b220-727e25cc2062 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:13:57.689530 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-cd005184-2e86-413d-b220-727e25cc2062 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:13:57.852305 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-cd005184-2e86-413d-b220-727e25cc2062 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:13:57.859811 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-cd005184-2e86-413d-b220-727e25cc2062 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062" status: 200 len: 1266 microversion: 2.1 time: 0.182868 Jul 19 11:13:57.865124 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 612/1230] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:13:57 2022] GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062 => generated 1266 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:13:58.878030 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-d6c91bb8-9c79-47e2-8ca2-58fa2da4dadf tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:13:58.878757 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-d6c91bb8-9c79-47e2-8ca2-58fa2da4dadf tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: f3c3d260-c061-48b7-8233-a3c3ef00d062] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:13:58.885184 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d6c91bb8-9c79-47e2-8ca2-58fa2da4dadf tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:13:58.885644 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d6c91bb8-9c79-47e2-8ca2-58fa2da4dadf tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:13:58.886420 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d6c91bb8-9c79-47e2-8ca2-58fa2da4dadf tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:13:58.886863 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d6c91bb8-9c79-47e2-8ca2-58fa2da4dadf tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:13:59.025459 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-d6c91bb8-9c79-47e2-8ca2-58fa2da4dadf tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:13:59.034615 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-d6c91bb8-9c79-47e2-8ca2-58fa2da4dadf tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062" status: 200 len: 1266 microversion: 2.1 time: 0.161492 Jul 19 11:13:59.035221 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 619/1231] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:13:58 2022] GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062 => generated 1266 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:14:00.051016 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-90a5ca32-bcfd-4c28-ac2f-8f9bec08123c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:14:00.051618 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-90a5ca32-bcfd-4c28-ac2f-8f9bec08123c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: f3c3d260-c061-48b7-8233-a3c3ef00d062] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:14:00.058095 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-90a5ca32-bcfd-4c28-ac2f-8f9bec08123c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:00.058539 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-90a5ca32-bcfd-4c28-ac2f-8f9bec08123c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:00.059319 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-90a5ca32-bcfd-4c28-ac2f-8f9bec08123c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:00.059750 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-90a5ca32-bcfd-4c28-ac2f-8f9bec08123c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:00.188084 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-90a5ca32-bcfd-4c28-ac2f-8f9bec08123c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:14:00.195464 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-90a5ca32-bcfd-4c28-ac2f-8f9bec08123c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062" status: 200 len: 1453 microversion: 2.1 time: 0.148291 Jul 19 11:14:00.196027 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 613/1232] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:14:00 2022] GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062 => generated 1453 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:14:00.212024 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-cb3c7eac-f7b9-429f-9389-a73841e59e4a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:14:00.212622 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-cb3c7eac-f7b9-429f-9389-a73841e59e4a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: f3c3d260-c061-48b7-8233-a3c3ef00d062] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:14:00.219025 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cb3c7eac-f7b9-429f-9389-a73841e59e4a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:00.219602 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cb3c7eac-f7b9-429f-9389-a73841e59e4a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:00.220368 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cb3c7eac-f7b9-429f-9389-a73841e59e4a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:00.220821 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cb3c7eac-f7b9-429f-9389-a73841e59e4a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:00.358181 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-cb3c7eac-f7b9-429f-9389-a73841e59e4a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:14:00.366305 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-cb3c7eac-f7b9-429f-9389-a73841e59e4a tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062" status: 200 len: 1453 microversion: 2.1 time: 0.158162 Jul 19 11:14:00.366860 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 620/1233] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:14:00 2022] GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062 => generated 1453 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:14:02.359854 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-ce5455d0-4d12-4201-8dce-c30bbd391460 req-b67a3a9f-914d-4d86-a13a-59fcdf5a1d8f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f3c3d260-c061-48b7-8233-a3c3ef00d062", "tag": "0518c0aa-dba9-44fd-93b2-3630f121afee"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:14:02.368742 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-ce5455d0-4d12-4201-8dce-c30bbd391460 req-b67a3a9f-914d-4d86-a13a-59fcdf5a1d8f service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:02.369189 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-ce5455d0-4d12-4201-8dce-c30bbd391460 req-b67a3a9f-914d-4d86-a13a-59fcdf5a1d8f service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:02.380681 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-ce5455d0-4d12-4201-8dce-c30bbd391460 req-b67a3a9f-914d-4d86-a13a-59fcdf5a1d8f service nova] Creating event network-changed:0518c0aa-dba9-44fd-93b2-3630f121afee for instance f3c3d260-c061-48b7-8233-a3c3ef00d062 on hv-cinder-maste Jul 19 11:14:02.385700 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-ce5455d0-4d12-4201-8dce-c30bbd391460 req-b67a3a9f-914d-4d86-a13a-59fcdf5a1d8f service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029485 Jul 19 11:14:02.386402 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 614/1234] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:14:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:14:06.399294 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-1e03c907-59bc-43dc-9b44-6222ecd801f4 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1868464067", "imageRef": "3085883e-2289-4314-aa47-9c41d4368b02", "flavorRef": "452", "networks": [{"uuid": "c2bd2a07-350f-4ebf-bfca-5af583b3ac26"}]}} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:14:06.448915 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.network.neutron [None req-1e03c907-59bc-43dc-9b44-6222ecd801f4 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] validate_networks() for [('c2bd2a07-350f-4ebf-bfca-5af583b3ac26', None, None, None, None, None)] {{(pid=117848) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 19 11:14:06.790071 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-1e03c907-59bc-43dc-9b44-6222ecd801f4 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Flavor limits 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 19 11:14:06.790495 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-1e03c907-59bc-43dc-9b44-6222ecd801f4 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Image limits 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 19 11:14:06.790967 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-1e03c907-59bc-43dc-9b44-6222ecd801f4 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Flavor pref 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 19 11:14:06.791378 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-1e03c907-59bc-43dc-9b44-6222ecd801f4 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Image pref 0:0:0 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 19 11:14:06.791788 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.virt.hardware [None req-1e03c907-59bc-43dc-9b44-6222ecd801f4 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117848) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 19 11:14:06.801001 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1e03c907-59bc-43dc-9b44-6222ecd801f4 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:06.801447 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1e03c907-59bc-43dc-9b44-6222ecd801f4 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:06.802265 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1e03c907-59bc-43dc-9b44-6222ecd801f4 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:06.802700 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-1e03c907-59bc-43dc-9b44-6222ecd801f4 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:06.819349 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.quota [None req-1e03c907-59bc-43dc-9b44-6222ecd801f4 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Getting quotas for project c519c231d95d4e14b2db727ea22d3b50. Resources: {'cores', 'instances', 'ram'} {{(pid=117848) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:14:06.823313 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.quota [None req-1e03c907-59bc-43dc-9b44-6222ecd801f4 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Getting quotas for user 8e228b9868654a6499bce784a631ec2d and project c519c231d95d4e14b2db727ea22d3b50. Resources: {'cores', 'instances', 'ram'} {{(pid=117848) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:14:06.833943 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-1e03c907-59bc-43dc-9b44-6222ecd801f4 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Going to run 1 instances... {{(pid=117848) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 19 11:14:06.836138 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-1e03c907-59bc-43dc-9b44-6222ecd801f4 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: f97e8737-e68e-46b3-9dad-bb5e05dd3c8f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='3085883e-2289-4314-aa47-9c41d4368b02',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117848) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 19 11:14:06.881133 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-1e03c907-59bc-43dc-9b44-6222ecd801f4 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.485604 Jul 19 11:14:06.881776 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 621/1235] 10.222.0.33 () {60 vars in 1146 bytes} [Tue Jul 19 11:14:06 2022] POST /compute/v2.1/servers => generated 380 bytes in 487 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 19 11:14:06.892890 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-89b12aa1-da3b-4faa-9376-3e9dfdcce05d tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:14:06.893592 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-89b12aa1-da3b-4faa-9376-3e9dfdcce05d tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: f97e8737-e68e-46b3-9dad-bb5e05dd3c8f] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:14:06.955622 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-89b12aa1-da3b-4faa-9376-3e9dfdcce05d tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e228b9868654a6499bce784a631ec2d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c519c231d95d4e14b2db727ea22d3b50', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:14:06.962351 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-89b12aa1-da3b-4faa-9376-3e9dfdcce05d tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "GET /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f" status: 200 len: 1219 microversion: 2.1 time: 0.074286 Jul 19 11:14:06.962915 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 615/1236] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:14:06 2022] GET /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f => generated 1219 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:14:07.980428 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-dc72963d-5364-4f45-848d-4732830ec28c tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:14:07.981455 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-dc72963d-5364-4f45-848d-4732830ec28c tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: f97e8737-e68e-46b3-9dad-bb5e05dd3c8f] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:14:07.989132 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dc72963d-5364-4f45-848d-4732830ec28c tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:07.989587 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dc72963d-5364-4f45-848d-4732830ec28c tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:07.990360 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dc72963d-5364-4f45-848d-4732830ec28c tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:07.990837 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dc72963d-5364-4f45-848d-4732830ec28c tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:08.063921 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-dc72963d-5364-4f45-848d-4732830ec28c tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e228b9868654a6499bce784a631ec2d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c519c231d95d4e14b2db727ea22d3b50', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:14:08.071358 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-dc72963d-5364-4f45-848d-4732830ec28c tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "GET /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f" status: 200 len: 1277 microversion: 2.1 time: 0.096048 Jul 19 11:14:08.072416 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 622/1237] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:14:07 2022] GET /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f => generated 1277 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:14:09.088249 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-1a391393-93eb-47a3-8210-932c378e971d tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:14:09.088900 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-1a391393-93eb-47a3-8210-932c378e971d tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: f97e8737-e68e-46b3-9dad-bb5e05dd3c8f] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:14:09.096247 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1a391393-93eb-47a3-8210-932c378e971d tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:09.096785 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1a391393-93eb-47a3-8210-932c378e971d tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:09.097567 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1a391393-93eb-47a3-8210-932c378e971d tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:09.098008 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1a391393-93eb-47a3-8210-932c378e971d tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:09.150457 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-93a1d855-f638-46ba-82ed-4845ad9009bb req-e9e8eb40-aa60-4c37-b677-fc08a7cff323 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f97e8737-e68e-46b3-9dad-bb5e05dd3c8f", "tag": "2b477480-72b8-46d1-8ee2-5e0a3d26d98a"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:14:09.160686 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-93a1d855-f638-46ba-82ed-4845ad9009bb req-e9e8eb40-aa60-4c37-b677-fc08a7cff323 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:09.161399 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-93a1d855-f638-46ba-82ed-4845ad9009bb req-e9e8eb40-aa60-4c37-b677-fc08a7cff323 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:09.175936 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-93a1d855-f638-46ba-82ed-4845ad9009bb req-e9e8eb40-aa60-4c37-b677-fc08a7cff323 service nova] Creating event network-changed:2b477480-72b8-46d1-8ee2-5e0a3d26d98a for instance f97e8737-e68e-46b3-9dad-bb5e05dd3c8f on hv-cinder-maste Jul 19 11:14:09.181028 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-93a1d855-f638-46ba-82ed-4845ad9009bb req-e9e8eb40-aa60-4c37-b677-fc08a7cff323 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034540 Jul 19 11:14:09.182240 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 623/1238] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:14:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:14:09.236214 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-1a391393-93eb-47a3-8210-932c378e971d tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e228b9868654a6499bce784a631ec2d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c519c231d95d4e14b2db727ea22d3b50', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:14:09.244132 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-1a391393-93eb-47a3-8210-932c378e971d tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "GET /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f" status: 200 len: 1319 microversion: 2.1 time: 0.160038 Jul 19 11:14:09.244721 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 616/1239] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:14:09 2022] GET /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f => generated 1319 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:14:10.263143 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-15a440db-b8b5-4684-9090-bfc56796b49b tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:14:10.263838 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-15a440db-b8b5-4684-9090-bfc56796b49b tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: f97e8737-e68e-46b3-9dad-bb5e05dd3c8f] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:14:10.270621 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-15a440db-b8b5-4684-9090-bfc56796b49b tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:10.271073 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-15a440db-b8b5-4684-9090-bfc56796b49b tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:10.271829 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-15a440db-b8b5-4684-9090-bfc56796b49b tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:10.272272 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-15a440db-b8b5-4684-9090-bfc56796b49b tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:10.407300 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-15a440db-b8b5-4684-9090-bfc56796b49b tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e228b9868654a6499bce784a631ec2d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c519c231d95d4e14b2db727ea22d3b50', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:14:10.414832 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-15a440db-b8b5-4684-9090-bfc56796b49b tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "GET /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f" status: 200 len: 1319 microversion: 2.1 time: 0.156479 Jul 19 11:14:10.415405 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 624/1240] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:14:10 2022] GET /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f => generated 1319 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:14:11.431400 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-19562851-b5ed-44a4-8437-43f41a7c38f6 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:14:11.432042 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-19562851-b5ed-44a4-8437-43f41a7c38f6 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: f97e8737-e68e-46b3-9dad-bb5e05dd3c8f] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:14:11.438520 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-19562851-b5ed-44a4-8437-43f41a7c38f6 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:11.438970 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-19562851-b5ed-44a4-8437-43f41a7c38f6 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:11.439737 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-19562851-b5ed-44a4-8437-43f41a7c38f6 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:11.440179 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-19562851-b5ed-44a4-8437-43f41a7c38f6 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:11.568687 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-19562851-b5ed-44a4-8437-43f41a7c38f6 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e228b9868654a6499bce784a631ec2d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c519c231d95d4e14b2db727ea22d3b50', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:14:11.576239 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-19562851-b5ed-44a4-8437-43f41a7c38f6 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "GET /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f" status: 200 len: 1319 microversion: 2.1 time: 0.148704 Jul 19 11:14:11.576854 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 617/1241] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:14:11 2022] GET /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f => generated 1319 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:14:12.593935 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-4923e470-7b1b-4b30-bdfb-0b801ef2fd25 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:14:12.594575 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-4923e470-7b1b-4b30-bdfb-0b801ef2fd25 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: f97e8737-e68e-46b3-9dad-bb5e05dd3c8f] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:14:12.601578 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4923e470-7b1b-4b30-bdfb-0b801ef2fd25 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:12.602029 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4923e470-7b1b-4b30-bdfb-0b801ef2fd25 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:12.602787 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4923e470-7b1b-4b30-bdfb-0b801ef2fd25 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:12.603220 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4923e470-7b1b-4b30-bdfb-0b801ef2fd25 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:12.733628 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-4923e470-7b1b-4b30-bdfb-0b801ef2fd25 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e228b9868654a6499bce784a631ec2d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c519c231d95d4e14b2db727ea22d3b50', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:14:12.740882 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-4923e470-7b1b-4b30-bdfb-0b801ef2fd25 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "GET /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f" status: 200 len: 1319 microversion: 2.1 time: 0.150829 Jul 19 11:14:12.741447 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 625/1242] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:14:12 2022] GET /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f => generated 1319 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:14:13.758346 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-ad5fb97c-0936-4280-bf90-a43c738171bc tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:14:13.758970 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-ad5fb97c-0936-4280-bf90-a43c738171bc tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: f97e8737-e68e-46b3-9dad-bb5e05dd3c8f] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:14:13.767338 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ad5fb97c-0936-4280-bf90-a43c738171bc tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:13.767809 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ad5fb97c-0936-4280-bf90-a43c738171bc tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:13.768561 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ad5fb97c-0936-4280-bf90-a43c738171bc tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:13.769022 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ad5fb97c-0936-4280-bf90-a43c738171bc tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:13.903610 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-ad5fb97c-0936-4280-bf90-a43c738171bc tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e228b9868654a6499bce784a631ec2d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c519c231d95d4e14b2db727ea22d3b50', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:14:13.910942 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-ad5fb97c-0936-4280-bf90-a43c738171bc tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "GET /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f" status: 200 len: 1319 microversion: 2.1 time: 0.156452 Jul 19 11:14:13.911508 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 618/1243] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:14:13 2022] GET /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f => generated 1319 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:14:14.927805 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-2b78db8f-cbb4-45b6-9d4f-a7c00d67ba86 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:14:14.928383 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-2b78db8f-cbb4-45b6-9d4f-a7c00d67ba86 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: f97e8737-e68e-46b3-9dad-bb5e05dd3c8f] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:14:14.930814 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_api:SELECT=42,nova_api:INSERT=3 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:14:14.936151 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2b78db8f-cbb4-45b6-9d4f-a7c00d67ba86 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:14.936596 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2b78db8f-cbb4-45b6-9d4f-a7c00d67ba86 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:14.937385 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2b78db8f-cbb4-45b6-9d4f-a7c00d67ba86 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:14.937816 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2b78db8f-cbb4-45b6-9d4f-a7c00d67ba86 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:14.941849 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=67 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:14:15.063474 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-2b78db8f-cbb4-45b6-9d4f-a7c00d67ba86 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e228b9868654a6499bce784a631ec2d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c519c231d95d4e14b2db727ea22d3b50', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:14:15.071339 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-2b78db8f-cbb4-45b6-9d4f-a7c00d67ba86 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "GET /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f" status: 200 len: 1319 microversion: 2.1 time: 0.147882 Jul 19 11:14:15.071905 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 626/1244] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:14:14 2022] GET /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f => generated 1319 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:14:15.748406 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-3cd8209d-2e16-46f9-8a05-3bdc80a71fa1 req-15e94754-519c-4a8e-8c85-cc9459fb6abe service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f97e8737-e68e-46b3-9dad-bb5e05dd3c8f", "name": "network-vif-plugged", "status": "completed", "tag": "2b477480-72b8-46d1-8ee2-5e0a3d26d98a"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:14:15.752987 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_api:SELECT=44,nova_api:INSERT=3 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:14:15.759000 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-3cd8209d-2e16-46f9-8a05-3bdc80a71fa1 req-15e94754-519c-4a8e-8c85-cc9459fb6abe service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:15.759449 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-3cd8209d-2e16-46f9-8a05-3bdc80a71fa1 req-15e94754-519c-4a8e-8c85-cc9459fb6abe service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:15.763740 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=67 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:14:15.772194 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-3cd8209d-2e16-46f9-8a05-3bdc80a71fa1 req-15e94754-519c-4a8e-8c85-cc9459fb6abe service nova] Creating event network-vif-plugged:2b477480-72b8-46d1-8ee2-5e0a3d26d98a for instance f97e8737-e68e-46b3-9dad-bb5e05dd3c8f on hv-cinder-maste Jul 19 11:14:15.777125 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-3cd8209d-2e16-46f9-8a05-3bdc80a71fa1 req-15e94754-519c-4a8e-8c85-cc9459fb6abe service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032355 Jul 19 11:14:15.777782 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 619/1245] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:14:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:14:16.087324 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-d36a3513-f004-40a9-b788-f0170ede26cc tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:14:16.087924 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-d36a3513-f004-40a9-b788-f0170ede26cc tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: f97e8737-e68e-46b3-9dad-bb5e05dd3c8f] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:14:16.094271 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d36a3513-f004-40a9-b788-f0170ede26cc tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:16.094709 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d36a3513-f004-40a9-b788-f0170ede26cc tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:16.095512 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d36a3513-f004-40a9-b788-f0170ede26cc tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:16.096394 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-d36a3513-f004-40a9-b788-f0170ede26cc tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:16.220378 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-d36a3513-f004-40a9-b788-f0170ede26cc tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e228b9868654a6499bce784a631ec2d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c519c231d95d4e14b2db727ea22d3b50', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:14:16.228270 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-d36a3513-f004-40a9-b788-f0170ede26cc tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "GET /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f" status: 200 len: 1319 microversion: 2.1 time: 0.144815 Jul 19 11:14:16.228853 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 627/1246] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:14:16 2022] GET /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f => generated 1319 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:14:17.243338 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell0:SELECT=2 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:14:17.247684 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-ee175bdf-2e33-4c9a-880c-67587a84a497 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:14:17.248376 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-ee175bdf-2e33-4c9a-880c-67587a84a497 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: f97e8737-e68e-46b3-9dad-bb5e05dd3c8f] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:14:17.255710 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ee175bdf-2e33-4c9a-880c-67587a84a497 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:17.256157 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ee175bdf-2e33-4c9a-880c-67587a84a497 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:17.256944 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ee175bdf-2e33-4c9a-880c-67587a84a497 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:17.257382 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-ee175bdf-2e33-4c9a-880c-67587a84a497 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:17.382700 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-ee175bdf-2e33-4c9a-880c-67587a84a497 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e228b9868654a6499bce784a631ec2d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c519c231d95d4e14b2db727ea22d3b50', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:14:17.390075 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-ee175bdf-2e33-4c9a-880c-67587a84a497 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "GET /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f" status: 200 len: 1319 microversion: 2.1 time: 0.148138 Jul 19 11:14:17.390648 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 620/1247] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:14:17 2022] GET /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f => generated 1319 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:14:18.406316 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-dd85cf8e-f9c9-4575-88ed-852e213a3fd6 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:14:18.406927 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-dd85cf8e-f9c9-4575-88ed-852e213a3fd6 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: f97e8737-e68e-46b3-9dad-bb5e05dd3c8f] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:14:18.409339 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell0:SELECT=2 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:14:18.410014 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell0:SELECT=6 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:14:18.416270 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dd85cf8e-f9c9-4575-88ed-852e213a3fd6 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:18.416750 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dd85cf8e-f9c9-4575-88ed-852e213a3fd6 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:18.417518 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dd85cf8e-f9c9-4575-88ed-852e213a3fd6 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:18.417976 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-dd85cf8e-f9c9-4575-88ed-852e213a3fd6 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:18.547210 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-dd85cf8e-f9c9-4575-88ed-852e213a3fd6 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e228b9868654a6499bce784a631ec2d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c519c231d95d4e14b2db727ea22d3b50', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:14:18.555198 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-dd85cf8e-f9c9-4575-88ed-852e213a3fd6 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "GET /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f" status: 200 len: 1508 microversion: 2.1 time: 0.152832 Jul 19 11:14:18.555806 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 628/1248] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:14:18 2022] GET /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f => generated 1508 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:14:33.824161 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-6cf580e3-eb65-4645-8cfd-6d3333241d8a tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "8260d192-c45d-4d7e-9349-fbd0f29ffc43", "device": "/dev/sdb"}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:14:33.825735 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-6cf580e3-eb65-4645-8cfd-6d3333241d8a tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: f97e8737-e68e-46b3-9dad-bb5e05dd3c8f] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:14:33.827658 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_api:SELECT=3 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:14:33.828350 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=7 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:14:33.834331 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6cf580e3-eb65-4645-8cfd-6d3333241d8a tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:33.834775 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6cf580e3-eb65-4645-8cfd-6d3333241d8a tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:33.835529 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6cf580e3-eb65-4645-8cfd-6d3333241d8a tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:33.835962 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6cf580e3-eb65-4645-8cfd-6d3333241d8a tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:34.475574 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-6cf580e3-eb65-4645-8cfd-6d3333241d8a tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "POST /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.654969 Jul 19 11:14:34.476179 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 621/1249] 10.222.0.33 () {60 vars in 1322 bytes} [Tue Jul 19 11:14:33 2022] POST /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f/os-volume_attachments => generated 194 bytes in 657 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:14:41.224108 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 19 11:14:41.224743 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Tue Jul 19 11:14:41 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 19 11:14:57.063239 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-12500053-7979-4e8a-84c0-3a400f022850 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:14:57.063711 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-12500053-7979-4e8a-84c0-3a400f022850 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: f97e8737-e68e-46b3-9dad-bb5e05dd3c8f] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:14:57.065878 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_api:SELECT=5 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:14:57.066610 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=17 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:14:57.072088 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-12500053-7979-4e8a-84c0-3a400f022850 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:57.072530 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-12500053-7979-4e8a-84c0-3a400f022850 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:57.073319 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-12500053-7979-4e8a-84c0-3a400f022850 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:57.073751 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-12500053-7979-4e8a-84c0-3a400f022850 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:57.203905 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.objects.instance [None req-12500053-7979-4e8a-84c0-3a400f022850 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lazy-loading 'services' on Instance uuid f97e8737-e68e-46b3-9dad-bb5e05dd3c8f {{(pid=117848) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 19 11:14:57.276304 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-12500053-7979-4e8a-84c0-3a400f022850 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "DELETE /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f/os-volume_attachments/8260d192-c45d-4d7e-9349-fbd0f29ffc43" status: 202 len: 0 microversion: 2.1 time: 0.216815 Jul 19 11:14:57.276937 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 629/1250] 10.222.0.33 () {58 vars in 1415 bytes} [Tue Jul 19 11:14:57 2022] DELETE /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f/os-volume_attachments/8260d192-c45d-4d7e-9349-fbd0f29ffc43 => generated 0 bytes in 218 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 19 11:14:58.390168 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-1e395c72-48dd-47a0-860c-760abe1d322b tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:14:58.390744 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-1e395c72-48dd-47a0-860c-760abe1d322b tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: f97e8737-e68e-46b3-9dad-bb5e05dd3c8f] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:14:58.393655 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_api:SELECT=2 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:14:58.394309 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=11,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:14:58.400898 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1e395c72-48dd-47a0-860c-760abe1d322b tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:58.401343 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1e395c72-48dd-47a0-860c-760abe1d322b tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:58.402110 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1e395c72-48dd-47a0-860c-760abe1d322b tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:58.402520 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1e395c72-48dd-47a0-860c-760abe1d322b tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:58.428005 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-1e395c72-48dd-47a0-860c-760abe1d322b tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: f97e8737-e68e-46b3-9dad-bb5e05dd3c8f] Going to try to terminate instance {{(pid=117849) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 19 11:14:58.500238 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-1e395c72-48dd-47a0-860c-760abe1d322b tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "DELETE /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f" status: 204 len: 0 microversion: 2.1 time: 0.113941 Jul 19 11:14:58.500952 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 622/1251] 10.222.0.33 () {58 vars in 1238 bytes} [Tue Jul 19 11:14:58 2022] DELETE /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f => generated 0 bytes in 116 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 19 11:14:58.510469 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-48b56ec7-2586-4f8d-82b4-83b3dba732ec tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:14:58.511201 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-48b56ec7-2586-4f8d-82b4-83b3dba732ec tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: f97e8737-e68e-46b3-9dad-bb5e05dd3c8f] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:14:58.517907 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-48b56ec7-2586-4f8d-82b4-83b3dba732ec tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:58.518353 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-48b56ec7-2586-4f8d-82b4-83b3dba732ec tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:58.519123 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-48b56ec7-2586-4f8d-82b4-83b3dba732ec tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:58.519560 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-48b56ec7-2586-4f8d-82b4-83b3dba732ec tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:58.646450 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-48b56ec7-2586-4f8d-82b4-83b3dba732ec tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e228b9868654a6499bce784a631ec2d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c519c231d95d4e14b2db727ea22d3b50', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:14:58.653885 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-48b56ec7-2586-4f8d-82b4-83b3dba732ec tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "GET /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f" status: 200 len: 1514 microversion: 2.1 time: 0.148203 Jul 19 11:14:58.654452 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 630/1252] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:14:58 2022] GET /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f => generated 1514 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:14:59.672178 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-6795674a-bad8-4682-9ca0-47359e7a1490 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:14:59.673492 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-6795674a-bad8-4682-9ca0-47359e7a1490 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: f97e8737-e68e-46b3-9dad-bb5e05dd3c8f] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:14:59.679868 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6795674a-bad8-4682-9ca0-47359e7a1490 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:59.680317 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6795674a-bad8-4682-9ca0-47359e7a1490 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:59.681112 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6795674a-bad8-4682-9ca0-47359e7a1490 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:14:59.681559 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-6795674a-bad8-4682-9ca0-47359e7a1490 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:14:59.814334 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-6795674a-bad8-4682-9ca0-47359e7a1490 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e228b9868654a6499bce784a631ec2d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c519c231d95d4e14b2db727ea22d3b50', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:14:59.821769 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-6795674a-bad8-4682-9ca0-47359e7a1490 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "GET /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f" status: 200 len: 1514 microversion: 2.1 time: 0.153809 Jul 19 11:14:59.822327 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 623/1253] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:14:59 2022] GET /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f => generated 1514 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:15:00.839533 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-e63c9fd9-06b6-47c0-9097-de146e0c254d tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:00.840122 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-e63c9fd9-06b6-47c0-9097-de146e0c254d tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: f97e8737-e68e-46b3-9dad-bb5e05dd3c8f] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:00.847686 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e63c9fd9-06b6-47c0-9097-de146e0c254d tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:00.848190 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e63c9fd9-06b6-47c0-9097-de146e0c254d tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:00.848982 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e63c9fd9-06b6-47c0-9097-de146e0c254d tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:00.849419 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e63c9fd9-06b6-47c0-9097-de146e0c254d tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:00.946664 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-e63c9fd9-06b6-47c0-9097-de146e0c254d tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e228b9868654a6499bce784a631ec2d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c519c231d95d4e14b2db727ea22d3b50', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:15:00.947030 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-9f32f057-ca8f-4e52-bd6e-73ec721ae2a0 req-83aa1250-dc52-4bbd-aba8-c20b6098f00d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f97e8737-e68e-46b3-9dad-bb5e05dd3c8f", "name": "network-vif-deleted", "tag": "2b477480-72b8-46d1-8ee2-5e0a3d26d98a"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:15:00.956822 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-e63c9fd9-06b6-47c0-9097-de146e0c254d tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "GET /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f" status: 200 len: 1472 microversion: 2.1 time: 0.120857 Jul 19 11:15:00.957279 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-9f32f057-ca8f-4e52-bd6e-73ec721ae2a0 req-83aa1250-dc52-4bbd-aba8-c20b6098f00d service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:00.957662 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-9f32f057-ca8f-4e52-bd6e-73ec721ae2a0 req-83aa1250-dc52-4bbd-aba8-c20b6098f00d service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:00.957818 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 631/1254] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:15:00 2022] GET /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f => generated 1472 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:15:00.968871 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-9f32f057-ca8f-4e52-bd6e-73ec721ae2a0 req-83aa1250-dc52-4bbd-aba8-c20b6098f00d service nova] Creating event network-vif-deleted:2b477480-72b8-46d1-8ee2-5e0a3d26d98a for instance f97e8737-e68e-46b3-9dad-bb5e05dd3c8f on hv-cinder-maste Jul 19 11:15:00.974309 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-9f32f057-ca8f-4e52-bd6e-73ec721ae2a0 req-83aa1250-dc52-4bbd-aba8-c20b6098f00d service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031074 Jul 19 11:15:00.974899 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 624/1255] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:15:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:15:01.974164 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-e1bf3a0c-255d-41ae-aba1-a6af5109a6f1 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:01.974765 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-e1bf3a0c-255d-41ae-aba1-a6af5109a6f1 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: f97e8737-e68e-46b3-9dad-bb5e05dd3c8f] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:01.981174 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e1bf3a0c-255d-41ae-aba1-a6af5109a6f1 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:01.981618 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e1bf3a0c-255d-41ae-aba1-a6af5109a6f1 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:01.982376 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e1bf3a0c-255d-41ae-aba1-a6af5109a6f1 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:01.982833 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-e1bf3a0c-255d-41ae-aba1-a6af5109a6f1 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:01.996363 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.wsgi [None req-e1bf3a0c-255d-41ae-aba1-a6af5109a6f1 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] HTTP exception thrown: Instance f97e8737-e68e-46b3-9dad-bb5e05dd3c8f could not be found. Jul 19 11:15:01.998001 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-e1bf3a0c-255d-41ae-aba1-a6af5109a6f1 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Request method failure captured: Jul 19 11:15:01.998001 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: request: GET /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f HTTP/1.1 Jul 19 11:15:01.998001 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept: application/json Jul 19 11:15:01.998001 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept-Encoding: identity Jul 19 11:15:01.998001 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Connection: close Jul 19 11:15:01.998001 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Length: 0 Jul 19 11:15:01.998001 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Type: application/json Jul 19 11:15:01.998001 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Host: 10.222.0.33 Jul 19 11:15:01.998001 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Openstack-System-Scope: None Jul 19 11:15:01.998001 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: User-Agent: python-urllib3/1.26.9 Jul 19 11:15:01.998001 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Auth-Token: gAAAAABi1pH8lyfocu9X1JPAORymcw6iMWQS5GQILGfG18_1gjd48cM0lKWmDRWFkkyCCVqSEyVedgrrad054F634JGUrerb-ztByhds9xBZFbOJmbOXxJLvmTg4yscuVJodZW0FbAmmPyV6nps5D2e3OfcNKTYlvHVO0srx315_D3X122e6yXo Jul 19 11:15:01.998001 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Id: None Jul 19 11:15:01.998001 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Name: None Jul 19 11:15:01.998001 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Identity-Status: Confirmed Jul 19 11:15:01.998001 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Is-Admin-Project: True Jul 19 11:15:01.998001 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Id: default Jul 19 11:15:01.998001 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Name: Default Jul 19 11:15:01.998001 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Id: c519c231d95d4e14b2db727ea22d3b50 Jul 19 11:15:01.998001 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1651191866 Jul 19 11:15:01.998001 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Role: member,reader Jul 19 11:15:01.998001 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Roles: member,reader Jul 19 11:15:01.999680 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/c519c231d95d4e14b2db727ea22d3b50"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/c519c231d95d4e14b2db727ea22d3b50"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/c519c231d95d4e14b2db727ea22d3b50"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_c519c231d95d4e14b2db727ea22d3b50"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:15:01.999680 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant: tempest-VolumesSnapshotTestJSON-1651191866 Jul 19 11:15:01.999680 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Id: c519c231d95d4e14b2db727ea22d3b50 Jul 19 11:15:01.999680 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1651191866 Jul 19 11:15:01.999680 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User: tempest-VolumesSnapshotTestJSON-1651191866-project Jul 19 11:15:01.999680 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Id: default Jul 19 11:15:01.999680 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Name: Default Jul 19 11:15:01.999680 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Id: 8e228b9868654a6499bce784a631ec2d Jul 19 11:15:01.999680 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Name: tempest-VolumesSnapshotTestJSON-1651191866-project Jul 19 11:15:01.999680 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: method: > Jul 19 11:15:01.999680 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: exception: Instance f97e8737-e68e-46b3-9dad-bb5e05dd3c8f could not be found. Jul 19 11:15:01.999680 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: action_args: {'id': 'f97e8737-e68e-46b3-9dad-bb5e05dd3c8f'} Jul 19 11:15:02.000721 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:15:02.000721 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:15:02.000721 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:15:02.000721 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:15:02.000721 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:15:02.000721 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:15:02.000721 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:15:02.000721 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:15:02.000721 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:15:02.000721 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:15:02.000721 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f97e8737-e68e-46b3-9dad-bb5e05dd3c8f could not be found. Jul 19 11:15:02.000721 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:15:02.000721 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:15:02.000721 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:15:02.000721 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:15:02.000721 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:15:02.000721 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:15:02.000721 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:15:02.000721 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:15:02.000721 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:15:02.002570 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:15:02.002570 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 19 11:15:02.002570 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 19 11:15:02.002570 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:15:02.002570 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:15:02.002570 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:15:02.002570 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:15:02.002570 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f97e8737-e68e-46b3-9dad-bb5e05dd3c8f could not be found. Jul 19 11:15:02.002570 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:15:02.002570 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:15:02.002570 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:15:02.002570 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:15:02.002570 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:15:02.002570 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:15:02.002570 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:15:02.002570 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:15:02.002570 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f97e8737-e68e-46b3-9dad-bb5e05dd3c8f could not be found. Jul 19 11:15:02.002570 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:15:02.004050 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-e1bf3a0c-255d-41ae-aba1-a6af5109a6f1 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Returning 404 to user: Instance f97e8737-e68e-46b3-9dad-bb5e05dd3c8f could not be found. {{(pid=117848) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:15:02.004050 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-e1bf3a0c-255d-41ae-aba1-a6af5109a6f1 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "GET /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f" status: 404 len: 111 microversion: 2.1 time: 0.029588 Jul 19 11:15:02.004050 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 632/1256] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:15:01 2022] GET /compute/v2.1/servers/f97e8737-e68e-46b3-9dad-bb5e05dd3c8f => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:15:09.740733 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-927628cd-21db-4835-b0dc-aec19b355c02 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1892722610", "imageRef": "3085883e-2289-4314-aa47-9c41d4368b02", "flavorRef": "452", "networks": [{"uuid": "c2bd2a07-350f-4ebf-bfca-5af583b3ac26"}]}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:15:09.790443 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.network.neutron [None req-927628cd-21db-4835-b0dc-aec19b355c02 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] validate_networks() for [('c2bd2a07-350f-4ebf-bfca-5af583b3ac26', None, None, None, None, None)] {{(pid=117849) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 19 11:15:10.197046 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-927628cd-21db-4835-b0dc-aec19b355c02 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Flavor limits 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 19 11:15:10.197464 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-927628cd-21db-4835-b0dc-aec19b355c02 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Image limits 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 19 11:15:10.197933 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-927628cd-21db-4835-b0dc-aec19b355c02 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Flavor pref 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 19 11:15:10.198337 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-927628cd-21db-4835-b0dc-aec19b355c02 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Image pref 0:0:0 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 19 11:15:10.198744 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.virt.hardware [None req-927628cd-21db-4835-b0dc-aec19b355c02 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117849) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 19 11:15:10.207488 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-927628cd-21db-4835-b0dc-aec19b355c02 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:10.207932 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-927628cd-21db-4835-b0dc-aec19b355c02 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:10.208752 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-927628cd-21db-4835-b0dc-aec19b355c02 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:10.209191 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-927628cd-21db-4835-b0dc-aec19b355c02 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:10.226009 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-927628cd-21db-4835-b0dc-aec19b355c02 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Getting quotas for project c519c231d95d4e14b2db727ea22d3b50. Resources: {'cores', 'instances', 'ram'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 19 11:15:10.230007 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.quota [None req-927628cd-21db-4835-b0dc-aec19b355c02 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Getting quotas for user 8e228b9868654a6499bce784a631ec2d and project c519c231d95d4e14b2db727ea22d3b50. Resources: {'cores', 'instances', 'ram'} {{(pid=117849) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 19 11:15:10.241833 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-927628cd-21db-4835-b0dc-aec19b355c02 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Going to run 1 instances... {{(pid=117849) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 19 11:15:10.244177 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-927628cd-21db-4835-b0dc-aec19b355c02 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: 11f9f6fe-f51f-4ac4-862d-93d028914265] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='3085883e-2289-4314-aa47-9c41d4368b02',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117849) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 19 11:15:10.288004 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-927628cd-21db-4835-b0dc-aec19b355c02 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.552020 Jul 19 11:15:10.288650 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 625/1257] 10.222.0.33 () {60 vars in 1146 bytes} [Tue Jul 19 11:15:09 2022] POST /compute/v2.1/servers => generated 380 bytes in 554 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 19 11:15:10.299077 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-80ecd492-1529-4308-84da-5e04c7377d01 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:10.299680 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-80ecd492-1529-4308-84da-5e04c7377d01 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: 11f9f6fe-f51f-4ac4-862d-93d028914265] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:10.352955 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-80ecd492-1529-4308-84da-5e04c7377d01 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e228b9868654a6499bce784a631ec2d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c519c231d95d4e14b2db727ea22d3b50', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:15:10.360591 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-80ecd492-1529-4308-84da-5e04c7377d01 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "GET /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265" status: 200 len: 1219 microversion: 2.1 time: 0.065256 Jul 19 11:15:10.361271 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 633/1258] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:15:10 2022] GET /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265 => generated 1219 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:15:11.379177 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-c822acca-f2a2-4164-9977-dbc9e9b2e1b4 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:11.379762 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-c822acca-f2a2-4164-9977-dbc9e9b2e1b4 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: 11f9f6fe-f51f-4ac4-862d-93d028914265] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:11.386731 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c822acca-f2a2-4164-9977-dbc9e9b2e1b4 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:11.387225 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c822acca-f2a2-4164-9977-dbc9e9b2e1b4 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:11.387989 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c822acca-f2a2-4164-9977-dbc9e9b2e1b4 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:11.388880 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-c822acca-f2a2-4164-9977-dbc9e9b2e1b4 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:11.452486 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-c822acca-f2a2-4164-9977-dbc9e9b2e1b4 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e228b9868654a6499bce784a631ec2d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c519c231d95d4e14b2db727ea22d3b50', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:15:11.460207 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-c822acca-f2a2-4164-9977-dbc9e9b2e1b4 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "GET /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265" status: 200 len: 1289 microversion: 2.1 time: 0.086292 Jul 19 11:15:11.461032 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 626/1259] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:15:11 2022] GET /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265 => generated 1289 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:15:12.477704 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-bd195ffa-9a7f-420d-99c3-64b4c74087ce tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:12.478320 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-bd195ffa-9a7f-420d-99c3-64b4c74087ce tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: 11f9f6fe-f51f-4ac4-862d-93d028914265] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:12.480771 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=23,nova_cell1:INSERT=1 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:15:12.487150 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-bd195ffa-9a7f-420d-99c3-64b4c74087ce tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:12.487611 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-bd195ffa-9a7f-420d-99c3-64b4c74087ce tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:12.488391 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-bd195ffa-9a7f-420d-99c3-64b4c74087ce tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:12.488860 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-bd195ffa-9a7f-420d-99c3-64b4c74087ce tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:12.699164 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-bd195ffa-9a7f-420d-99c3-64b4c74087ce tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e228b9868654a6499bce784a631ec2d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c519c231d95d4e14b2db727ea22d3b50', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:15:12.708090 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-bd195ffa-9a7f-420d-99c3-64b4c74087ce tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "GET /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265" status: 200 len: 1319 microversion: 2.1 time: 0.234292 Jul 19 11:15:12.708672 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 634/1260] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:15:12 2022] GET /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265 => generated 1319 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:15:12.719101 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-0003853f-d0f1-4dfa-bbcc-fad5b18ab9be req-8d2645f8-636c-4823-8572-fa7eb92de0d7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "11f9f6fe-f51f-4ac4-862d-93d028914265", "tag": "71fc0996-d492-4438-bd06-b573505fa104"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:15:12.728547 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-0003853f-d0f1-4dfa-bbcc-fad5b18ab9be req-8d2645f8-636c-4823-8572-fa7eb92de0d7 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:12.729119 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-0003853f-d0f1-4dfa-bbcc-fad5b18ab9be req-8d2645f8-636c-4823-8572-fa7eb92de0d7 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:12.741379 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-0003853f-d0f1-4dfa-bbcc-fad5b18ab9be req-8d2645f8-636c-4823-8572-fa7eb92de0d7 service nova] Creating event network-changed:71fc0996-d492-4438-bd06-b573505fa104 for instance 11f9f6fe-f51f-4ac4-862d-93d028914265 on hv-cinder-maste Jul 19 11:15:12.746277 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-0003853f-d0f1-4dfa-bbcc-fad5b18ab9be req-8d2645f8-636c-4823-8572-fa7eb92de0d7 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030982 Jul 19 11:15:12.746935 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 627/1261] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:15:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:15:13.725353 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-5a9e4676-40b7-4286-9827-b4f80e285cec tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:13.725973 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-5a9e4676-40b7-4286-9827-b4f80e285cec tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: 11f9f6fe-f51f-4ac4-862d-93d028914265] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:13.732516 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5a9e4676-40b7-4286-9827-b4f80e285cec tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:13.733010 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5a9e4676-40b7-4286-9827-b4f80e285cec tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:13.733772 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5a9e4676-40b7-4286-9827-b4f80e285cec tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:13.734203 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5a9e4676-40b7-4286-9827-b4f80e285cec tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:13.861531 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-5a9e4676-40b7-4286-9827-b4f80e285cec tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e228b9868654a6499bce784a631ec2d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c519c231d95d4e14b2db727ea22d3b50', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:15:13.868813 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-5a9e4676-40b7-4286-9827-b4f80e285cec tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "GET /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265" status: 200 len: 1319 microversion: 2.1 time: 0.147300 Jul 19 11:15:13.869368 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 635/1262] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:15:13 2022] GET /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265 => generated 1319 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:15:14.887031 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-5660bc37-51ed-4eca-b404-0fd8eb3ed839 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:14.887722 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-5660bc37-51ed-4eca-b404-0fd8eb3ed839 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: 11f9f6fe-f51f-4ac4-862d-93d028914265] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:14.894637 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5660bc37-51ed-4eca-b404-0fd8eb3ed839 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:14.895085 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5660bc37-51ed-4eca-b404-0fd8eb3ed839 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:14.895841 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5660bc37-51ed-4eca-b404-0fd8eb3ed839 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:14.896281 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5660bc37-51ed-4eca-b404-0fd8eb3ed839 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:15.028457 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-5660bc37-51ed-4eca-b404-0fd8eb3ed839 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e228b9868654a6499bce784a631ec2d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c519c231d95d4e14b2db727ea22d3b50', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:15:15.035785 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-5660bc37-51ed-4eca-b404-0fd8eb3ed839 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "GET /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265" status: 200 len: 1319 microversion: 2.1 time: 0.153581 Jul 19 11:15:15.036351 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 628/1263] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:15:14 2022] GET /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265 => generated 1319 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:15:16.052888 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-4578ab63-698b-4310-8280-7964156e4810 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:16.053479 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-4578ab63-698b-4310-8280-7964156e4810 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: 11f9f6fe-f51f-4ac4-862d-93d028914265] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:16.060115 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4578ab63-698b-4310-8280-7964156e4810 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:16.060488 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4578ab63-698b-4310-8280-7964156e4810 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:16.061278 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4578ab63-698b-4310-8280-7964156e4810 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:16.061707 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-4578ab63-698b-4310-8280-7964156e4810 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:16.192470 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-4578ab63-698b-4310-8280-7964156e4810 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e228b9868654a6499bce784a631ec2d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c519c231d95d4e14b2db727ea22d3b50', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:15:16.199759 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-4578ab63-698b-4310-8280-7964156e4810 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "GET /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265" status: 200 len: 1319 microversion: 2.1 time: 0.150808 Jul 19 11:15:16.200320 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 636/1264] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:15:16 2022] GET /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265 => generated 1319 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:15:17.217235 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-1d3265a1-4386-4d37-80f1-be1f7751c61e tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:17.217809 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-1d3265a1-4386-4d37-80f1-be1f7751c61e tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: 11f9f6fe-f51f-4ac4-862d-93d028914265] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:17.224220 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1d3265a1-4386-4d37-80f1-be1f7751c61e tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:17.224679 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1d3265a1-4386-4d37-80f1-be1f7751c61e tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:17.225473 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1d3265a1-4386-4d37-80f1-be1f7751c61e tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:17.225921 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-1d3265a1-4386-4d37-80f1-be1f7751c61e tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:17.350491 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-1d3265a1-4386-4d37-80f1-be1f7751c61e tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e228b9868654a6499bce784a631ec2d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c519c231d95d4e14b2db727ea22d3b50', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:15:17.358095 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-1d3265a1-4386-4d37-80f1-be1f7751c61e tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "GET /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265" status: 200 len: 1319 microversion: 2.1 time: 0.144763 Jul 19 11:15:17.358659 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 629/1265] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:15:17 2022] GET /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265 => generated 1319 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:15:18.033041 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-7b535e4e-b5ae-4063-9f6a-3110246808f2 req-fe73b3dc-f45e-44e8-a1d3-ef157d916c5d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "11f9f6fe-f51f-4ac4-862d-93d028914265", "name": "network-vif-plugged", "status": "completed", "tag": "71fc0996-d492-4438-bd06-b573505fa104"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:15:18.041454 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-7b535e4e-b5ae-4063-9f6a-3110246808f2 req-fe73b3dc-f45e-44e8-a1d3-ef157d916c5d service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:18.041936 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-7b535e4e-b5ae-4063-9f6a-3110246808f2 req-fe73b3dc-f45e-44e8-a1d3-ef157d916c5d service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:18.052771 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-7b535e4e-b5ae-4063-9f6a-3110246808f2 req-fe73b3dc-f45e-44e8-a1d3-ef157d916c5d service nova] Creating event network-vif-plugged:71fc0996-d492-4438-bd06-b573505fa104 for instance 11f9f6fe-f51f-4ac4-862d-93d028914265 on hv-cinder-maste Jul 19 11:15:18.057726 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-7b535e4e-b5ae-4063-9f6a-3110246808f2 req-fe73b3dc-f45e-44e8-a1d3-ef157d916c5d service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.089443 Jul 19 11:15:18.058334 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 637/1266] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:15:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:15:18.374768 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-de92e0df-3814-4f1c-9f37-cf43a75f7c5a tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:18.375801 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-de92e0df-3814-4f1c-9f37-cf43a75f7c5a tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: 11f9f6fe-f51f-4ac4-862d-93d028914265] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:18.382500 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-de92e0df-3814-4f1c-9f37-cf43a75f7c5a tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:18.382946 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-de92e0df-3814-4f1c-9f37-cf43a75f7c5a tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:18.383701 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-de92e0df-3814-4f1c-9f37-cf43a75f7c5a tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:18.384135 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-de92e0df-3814-4f1c-9f37-cf43a75f7c5a tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:18.510055 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-de92e0df-3814-4f1c-9f37-cf43a75f7c5a tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e228b9868654a6499bce784a631ec2d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c519c231d95d4e14b2db727ea22d3b50', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:15:18.517929 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-de92e0df-3814-4f1c-9f37-cf43a75f7c5a tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "GET /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265" status: 200 len: 1319 microversion: 2.1 time: 0.147078 Jul 19 11:15:18.518507 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 630/1267] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:15:18 2022] GET /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265 => generated 1319 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:15:19.536151 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-07561305-3c11-4546-b825-1d4f736d8c9d tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:19.536870 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-07561305-3c11-4546-b825-1d4f736d8c9d tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: 11f9f6fe-f51f-4ac4-862d-93d028914265] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:19.543094 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-07561305-3c11-4546-b825-1d4f736d8c9d tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:19.543542 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-07561305-3c11-4546-b825-1d4f736d8c9d tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:19.544296 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-07561305-3c11-4546-b825-1d4f736d8c9d tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:19.544751 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-07561305-3c11-4546-b825-1d4f736d8c9d tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:19.677513 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-07561305-3c11-4546-b825-1d4f736d8c9d tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e228b9868654a6499bce784a631ec2d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c519c231d95d4e14b2db727ea22d3b50', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:15:19.685939 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-07561305-3c11-4546-b825-1d4f736d8c9d tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "GET /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265" status: 200 len: 1508 microversion: 2.1 time: 0.154710 Jul 19 11:15:19.686695 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 638/1268] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:15:19 2022] GET /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265 => generated 1508 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:15:19.702505 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-297a28d2-9c18-4ecb-b6a7-1da6a45db480 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "3bf0ae7a-ab3a-4531-8a2f-7c0c2ffaa1d3", "device": "/dev/sdb"}} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:15:19.704111 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-297a28d2-9c18-4ecb-b6a7-1da6a45db480 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: 11f9f6fe-f51f-4ac4-862d-93d028914265] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:19.710567 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-297a28d2-9c18-4ecb-b6a7-1da6a45db480 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:19.711014 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-297a28d2-9c18-4ecb-b6a7-1da6a45db480 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:19.711777 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-297a28d2-9c18-4ecb-b6a7-1da6a45db480 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:19.712211 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-297a28d2-9c18-4ecb-b6a7-1da6a45db480 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:19.958033 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-297a28d2-9c18-4ecb-b6a7-1da6a45db480 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "POST /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.259287 Jul 19 11:15:19.963267 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 631/1269] 10.222.0.33 () {60 vars in 1322 bytes} [Tue Jul 19 11:15:19 2022] POST /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265/os-volume_attachments => generated 194 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:15:31.446071 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell0:SELECT=2 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:15:31.446995 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell0:SELECT=6 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:15:31.447520 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_api:SELECT=36,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:15:31.448254 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG dbcounter [-] [117849] Writing DB stats nova_cell1:SELECT=61,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=117849) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:15:31.455946 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-2d2fbce1-90d0-4693-93ec-91b7eb749338 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:31.456444 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-2d2fbce1-90d0-4693-93ec-91b7eb749338 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: f3c3d260-c061-48b7-8233-a3c3ef00d062] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:31.459208 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell0:SELECT=2 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:15:31.459403 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_api:SELECT=22 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:15:31.460035 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG dbcounter [-] [117848] Writing DB stats nova_cell1:SELECT=26 {{(pid=117848) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 11:15:31.461675 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-77895da5-ae3c-498e-b941-4f4f515b4231 req-b200e198-2b6f-4764-a442-f8483fb833f3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f3c3d260-c061-48b7-8233-a3c3ef00d062", "tag": "0518c0aa-dba9-44fd-93b2-3630f121afee"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:15:31.478295 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2d2fbce1-90d0-4693-93ec-91b7eb749338 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:31.478735 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2d2fbce1-90d0-4693-93ec-91b7eb749338 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:31.479501 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2d2fbce1-90d0-4693-93ec-91b7eb749338 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:31.479930 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-2d2fbce1-90d0-4693-93ec-91b7eb749338 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:31.484581 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-77895da5-ae3c-498e-b941-4f4f515b4231 req-b200e198-2b6f-4764-a442-f8483fb833f3 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:31.485063 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-77895da5-ae3c-498e-b941-4f4f515b4231 req-b200e198-2b6f-4764-a442-f8483fb833f3 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:31.498391 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-77895da5-ae3c-498e-b941-4f4f515b4231 req-b200e198-2b6f-4764-a442-f8483fb833f3 service nova] Creating event network-changed:0518c0aa-dba9-44fd-93b2-3630f121afee for instance f3c3d260-c061-48b7-8233-a3c3ef00d062 on hv-cinder-maste Jul 19 11:15:31.503894 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-77895da5-ae3c-498e-b941-4f4f515b4231 req-b200e198-2b6f-4764-a442-f8483fb833f3 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.058940 Jul 19 11:15:31.504823 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 632/1270] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:15:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:15:31.514178 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-2d2fbce1-90d0-4693-93ec-91b7eb749338 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: f3c3d260-c061-48b7-8233-a3c3ef00d062] Going to try to terminate instance {{(pid=117848) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 19 11:15:31.596819 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-2d2fbce1-90d0-4693-93ec-91b7eb749338 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "DELETE /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062" status: 204 len: 0 microversion: 2.1 time: 0.154290 Jul 19 11:15:31.601361 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 639/1271] 10.222.0.33 () {58 vars in 1238 bytes} [Tue Jul 19 11:15:31 2022] DELETE /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 19 11:15:31.606456 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-19a8f249-aa72-4cac-995e-e1edb3e3859e tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:31.608435 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-19a8f249-aa72-4cac-995e-e1edb3e3859e tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: f3c3d260-c061-48b7-8233-a3c3ef00d062] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:31.615633 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-19a8f249-aa72-4cac-995e-e1edb3e3859e tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:31.616191 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-19a8f249-aa72-4cac-995e-e1edb3e3859e tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:31.617042 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-19a8f249-aa72-4cac-995e-e1edb3e3859e tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:31.617549 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-19a8f249-aa72-4cac-995e-e1edb3e3859e tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:31.759594 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-19a8f249-aa72-4cac-995e-e1edb3e3859e tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:15:31.771497 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-19a8f249-aa72-4cac-995e-e1edb3e3859e tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062" status: 200 len: 1578 microversion: 2.1 time: 0.169075 Jul 19 11:15:31.772740 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 633/1272] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:15:31 2022] GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062 => generated 1578 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:15:32.790173 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-20c96bec-b0b8-465f-a867-77f975ca5ffa tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:32.790852 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-20c96bec-b0b8-465f-a867-77f975ca5ffa tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: f3c3d260-c061-48b7-8233-a3c3ef00d062] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:32.798172 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-20c96bec-b0b8-465f-a867-77f975ca5ffa tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:32.798627 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-20c96bec-b0b8-465f-a867-77f975ca5ffa tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:32.799399 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-20c96bec-b0b8-465f-a867-77f975ca5ffa tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:32.799838 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-20c96bec-b0b8-465f-a867-77f975ca5ffa tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:32.933937 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-20c96bec-b0b8-465f-a867-77f975ca5ffa tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:15:32.941524 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-20c96bec-b0b8-465f-a867-77f975ca5ffa tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062" status: 200 len: 1578 microversion: 2.1 time: 0.155680 Jul 19 11:15:32.942088 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 640/1273] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:15:32 2022] GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062 => generated 1578 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:15:33.958364 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-b7539170-c6dd-4bcc-b61f-2116c0cfb1e5 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:33.958964 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-b7539170-c6dd-4bcc-b61f-2116c0cfb1e5 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: f3c3d260-c061-48b7-8233-a3c3ef00d062] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:33.965418 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b7539170-c6dd-4bcc-b61f-2116c0cfb1e5 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:33.965862 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b7539170-c6dd-4bcc-b61f-2116c0cfb1e5 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:33.966611 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b7539170-c6dd-4bcc-b61f-2116c0cfb1e5 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:33.967040 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-b7539170-c6dd-4bcc-b61f-2116c0cfb1e5 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:34.093838 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-b7539170-c6dd-4bcc-b61f-2116c0cfb1e5 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:15:34.101246 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-b7539170-c6dd-4bcc-b61f-2116c0cfb1e5 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062" status: 200 len: 1578 microversion: 2.1 time: 0.146695 Jul 19 11:15:34.101803 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 634/1274] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:15:33 2022] GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062 => generated 1578 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:15:35.119731 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-cbda4cfa-8d62-4e37-8308-9d853d16a37b tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:35.120326 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-cbda4cfa-8d62-4e37-8308-9d853d16a37b tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: f3c3d260-c061-48b7-8233-a3c3ef00d062] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:35.127724 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cbda4cfa-8d62-4e37-8308-9d853d16a37b tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:35.128180 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cbda4cfa-8d62-4e37-8308-9d853d16a37b tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:35.128989 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cbda4cfa-8d62-4e37-8308-9d853d16a37b tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:35.129430 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-cbda4cfa-8d62-4e37-8308-9d853d16a37b tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:35.289224 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-cbda4cfa-8d62-4e37-8308-9d853d16a37b tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:15:35.297353 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-cbda4cfa-8d62-4e37-8308-9d853d16a37b tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062" status: 200 len: 1578 microversion: 2.1 time: 0.181569 Jul 19 11:15:35.297931 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 641/1275] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:15:35 2022] GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062 => generated 1578 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:15:36.140591 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-b9f8917a-f218-48bc-b342-60f8f3817708 req-6ae8f44f-19b3-4990-95bb-a5f866c96a1b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f3c3d260-c061-48b7-8233-a3c3ef00d062", "name": "network-vif-deleted", "tag": "0518c0aa-dba9-44fd-93b2-3630f121afee"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:15:36.148537 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-b9f8917a-f218-48bc-b342-60f8f3817708 req-6ae8f44f-19b3-4990-95bb-a5f866c96a1b service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:36.149125 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-b9f8917a-f218-48bc-b342-60f8f3817708 req-6ae8f44f-19b3-4990-95bb-a5f866c96a1b service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:36.160328 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-b9f8917a-f218-48bc-b342-60f8f3817708 req-6ae8f44f-19b3-4990-95bb-a5f866c96a1b service nova] Creating event network-vif-deleted:0518c0aa-dba9-44fd-93b2-3630f121afee for instance f3c3d260-c061-48b7-8233-a3c3ef00d062 on hv-cinder-maste Jul 19 11:15:36.165274 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-b9f8917a-f218-48bc-b342-60f8f3817708 req-6ae8f44f-19b3-4990-95bb-a5f866c96a1b service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.090913 Jul 19 11:15:36.165841 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 635/1276] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:15:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:15:36.314974 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-5a720817-e6b8-4d7d-b16b-47d188eeb23e tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:36.315502 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-5a720817-e6b8-4d7d-b16b-47d188eeb23e tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: f3c3d260-c061-48b7-8233-a3c3ef00d062] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:36.322150 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5a720817-e6b8-4d7d-b16b-47d188eeb23e tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:36.322594 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5a720817-e6b8-4d7d-b16b-47d188eeb23e tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:36.323350 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5a720817-e6b8-4d7d-b16b-47d188eeb23e tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:36.323777 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5a720817-e6b8-4d7d-b16b-47d188eeb23e tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:36.388729 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-5a720817-e6b8-4d7d-b16b-47d188eeb23e tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:15:36.400097 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-5a720817-e6b8-4d7d-b16b-47d188eeb23e tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062" status: 200 len: 1391 microversion: 2.1 time: 0.089802 Jul 19 11:15:36.400998 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 642/1277] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:15:36 2022] GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062 => generated 1391 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:15:37.417757 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-0167dde1-ddef-4ad8-934b-469f0a08571c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:37.418503 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-0167dde1-ddef-4ad8-934b-469f0a08571c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: f3c3d260-c061-48b7-8233-a3c3ef00d062] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:37.424652 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0167dde1-ddef-4ad8-934b-469f0a08571c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:37.425132 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0167dde1-ddef-4ad8-934b-469f0a08571c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:37.425890 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0167dde1-ddef-4ad8-934b-469f0a08571c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:37.426330 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-0167dde1-ddef-4ad8-934b-469f0a08571c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:37.439251 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.wsgi [None req-0167dde1-ddef-4ad8-934b-469f0a08571c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] HTTP exception thrown: Instance f3c3d260-c061-48b7-8233-a3c3ef00d062 could not be found. Jul 19 11:15:37.440892 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-0167dde1-ddef-4ad8-934b-469f0a08571c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Request method failure captured: Jul 19 11:15:37.440892 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: request: GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062 HTTP/1.1 Jul 19 11:15:37.440892 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept: application/json Jul 19 11:15:37.440892 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept-Encoding: identity Jul 19 11:15:37.440892 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Connection: close Jul 19 11:15:37.440892 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Length: 0 Jul 19 11:15:37.440892 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Type: application/json Jul 19 11:15:37.440892 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Host: 10.222.0.33 Jul 19 11:15:37.440892 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Openstack-System-Scope: None Jul 19 11:15:37.440892 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: User-Agent: python-urllib3/1.26.9 Jul 19 11:15:37.440892 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Auth-Token: gAAAAABi1pCKSZ-4SJ5s_ItAgtEK2gvn4DyrOW7x3YHwCqO5aRGl0Sxm4ZZRq5InRFnucVfPkLXEpzi5hBz6xHVKVr9nFez8ydrjVvbO97mMRi8k_yB3xxgwAy6BC0nPq6dyYILo3FmB20Si0ydiOAi4CYj6_z7hgsm3RTs6jezifIIUuhEeGdE Jul 19 11:15:37.440892 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Id: None Jul 19 11:15:37.440892 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Name: None Jul 19 11:15:37.440892 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Identity-Status: Confirmed Jul 19 11:15:37.440892 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Is-Admin-Project: True Jul 19 11:15:37.440892 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Id: default Jul 19 11:15:37.440892 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Name: Default Jul 19 11:15:37.440892 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Id: a7109adf88ac4391aee35c150e96e0b1 Jul 19 11:15:37.440892 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Name: tempest-TestVolumeBootPattern-1429943467 Jul 19 11:15:37.440892 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Role: member,reader Jul 19 11:15:37.440892 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Roles: member,reader Jul 19 11:15:37.442530 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7109adf88ac4391aee35c150e96e0b1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/a7109adf88ac4391aee35c150e96e0b1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7109adf88ac4391aee35c150e96e0b1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_a7109adf88ac4391aee35c150e96e0b1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:15:37.442530 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant: tempest-TestVolumeBootPattern-1429943467 Jul 19 11:15:37.442530 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Id: a7109adf88ac4391aee35c150e96e0b1 Jul 19 11:15:37.442530 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Name: tempest-TestVolumeBootPattern-1429943467 Jul 19 11:15:37.442530 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User: tempest-TestVolumeBootPattern-1429943467-project Jul 19 11:15:37.442530 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Id: default Jul 19 11:15:37.442530 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Name: Default Jul 19 11:15:37.442530 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Id: a217e983effa43f3a4f84221984189c5 Jul 19 11:15:37.442530 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Name: tempest-TestVolumeBootPattern-1429943467-project Jul 19 11:15:37.442530 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: method: > Jul 19 11:15:37.442530 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: exception: Instance f3c3d260-c061-48b7-8233-a3c3ef00d062 could not be found. Jul 19 11:15:37.442530 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: action_args: {'id': 'f3c3d260-c061-48b7-8233-a3c3ef00d062'} Jul 19 11:15:37.443670 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:15:37.443670 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:15:37.443670 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:15:37.443670 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:15:37.443670 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:15:37.443670 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:15:37.443670 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:15:37.443670 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:15:37.443670 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:15:37.443670 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:15:37.443670 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f3c3d260-c061-48b7-8233-a3c3ef00d062 could not be found. Jul 19 11:15:37.443670 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:15:37.443670 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:15:37.443670 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:15:37.443670 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:15:37.443670 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:15:37.443670 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:15:37.443670 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:15:37.443670 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:15:37.443670 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:15:37.445089 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:15:37.445089 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 19 11:15:37.445089 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 19 11:15:37.445089 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:15:37.445089 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:15:37.445089 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:15:37.445089 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:15:37.445089 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f3c3d260-c061-48b7-8233-a3c3ef00d062 could not be found. Jul 19 11:15:37.445089 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:15:37.445089 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:15:37.445089 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:15:37.445089 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:15:37.445089 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:15:37.445089 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:15:37.445089 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:15:37.445089 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:15:37.445089 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f3c3d260-c061-48b7-8233-a3c3ef00d062 could not be found. Jul 19 11:15:37.445089 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:15:37.447032 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-0167dde1-ddef-4ad8-934b-469f0a08571c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Returning 404 to user: Instance f3c3d260-c061-48b7-8233-a3c3ef00d062 could not be found. {{(pid=117849) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:15:37.447032 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-0167dde1-ddef-4ad8-934b-469f0a08571c tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062" status: 404 len: 111 microversion: 2.1 time: 0.029330 Jul 19 11:15:37.447032 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 636/1278] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:15:37 2022] GET /compute/v2.1/servers/f3c3d260-c061-48b7-8233-a3c3ef00d062 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:15:39.289208 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-47f51242-fedf-464b-b4b2-44e8fa9055b5 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:39.289707 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-47f51242-fedf-464b-b4b2-44e8fa9055b5 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: 11f9f6fe-f51f-4ac4-862d-93d028914265] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:39.295997 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-47f51242-fedf-464b-b4b2-44e8fa9055b5 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:39.296433 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-47f51242-fedf-464b-b4b2-44e8fa9055b5 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:39.297219 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-47f51242-fedf-464b-b4b2-44e8fa9055b5 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:39.297650 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-47f51242-fedf-464b-b4b2-44e8fa9055b5 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:39.377732 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.objects.instance [None req-47f51242-fedf-464b-b4b2-44e8fa9055b5 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lazy-loading 'services' on Instance uuid 11f9f6fe-f51f-4ac4-862d-93d028914265 {{(pid=117848) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 19 11:15:39.447861 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-47f51242-fedf-464b-b4b2-44e8fa9055b5 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "DELETE /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265/os-volume_attachments/3bf0ae7a-ab3a-4531-8a2f-7c0c2ffaa1d3" status: 202 len: 0 microversion: 2.1 time: 0.162400 Jul 19 11:15:39.448445 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 643/1279] 10.222.0.33 () {58 vars in 1415 bytes} [Tue Jul 19 11:15:39 2022] DELETE /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265/os-volume_attachments/3bf0ae7a-ab3a-4531-8a2f-7c0c2ffaa1d3 => generated 0 bytes in 164 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 19 11:15:40.558791 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-bafcc706-7711-4baa-809c-ad4af7b5ab9f tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:40.559355 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-bafcc706-7711-4baa-809c-ad4af7b5ab9f tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: 11f9f6fe-f51f-4ac4-862d-93d028914265] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:40.566374 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bafcc706-7711-4baa-809c-ad4af7b5ab9f tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:40.566820 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bafcc706-7711-4baa-809c-ad4af7b5ab9f tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:40.567580 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bafcc706-7711-4baa-809c-ad4af7b5ab9f tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:40.568015 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bafcc706-7711-4baa-809c-ad4af7b5ab9f tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:40.594788 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-bafcc706-7711-4baa-809c-ad4af7b5ab9f tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: 11f9f6fe-f51f-4ac4-862d-93d028914265] Going to try to terminate instance {{(pid=117849) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 19 11:15:40.668412 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-bafcc706-7711-4baa-809c-ad4af7b5ab9f tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "DELETE /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265" status: 204 len: 0 microversion: 2.1 time: 0.113186 Jul 19 11:15:40.669239 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 637/1280] 10.222.0.33 () {58 vars in 1238 bytes} [Tue Jul 19 11:15:40 2022] DELETE /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 19 11:15:40.679773 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-b8fb454f-edad-4eb4-8f88-b87a5b0add87 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:40.680510 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-b8fb454f-edad-4eb4-8f88-b87a5b0add87 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: 11f9f6fe-f51f-4ac4-862d-93d028914265] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:40.689431 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b8fb454f-edad-4eb4-8f88-b87a5b0add87 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:40.689891 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b8fb454f-edad-4eb4-8f88-b87a5b0add87 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:40.690696 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b8fb454f-edad-4eb4-8f88-b87a5b0add87 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:40.691133 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-b8fb454f-edad-4eb4-8f88-b87a5b0add87 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:40.803399 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-a9abeef2-49e1-4e2e-aeb6-74a9692ad1dc tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:40.803980 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-a9abeef2-49e1-4e2e-aeb6-74a9692ad1dc tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:40.810065 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a9abeef2-49e1-4e2e-aeb6-74a9692ad1dc tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:40.810518 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a9abeef2-49e1-4e2e-aeb6-74a9692ad1dc tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:40.811475 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a9abeef2-49e1-4e2e-aeb6-74a9692ad1dc tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:40.812885 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-a9abeef2-49e1-4e2e-aeb6-74a9692ad1dc tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:40.838130 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-a9abeef2-49e1-4e2e-aeb6-74a9692ad1dc tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Going to try to terminate instance {{(pid=117849) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 19 11:15:40.870601 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-b8fb454f-edad-4eb4-8f88-b87a5b0add87 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e228b9868654a6499bce784a631ec2d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c519c231d95d4e14b2db727ea22d3b50', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:15:40.879625 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-b8fb454f-edad-4eb4-8f88-b87a5b0add87 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "GET /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265" status: 200 len: 1514 microversion: 2.1 time: 0.205073 Jul 19 11:15:40.880910 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 644/1281] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:15:40 2022] GET /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265 => generated 1514 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:15:40.886651 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [req-23c9a14e-c59c-4b2f-b347-1190cd0da8e0 req-c4a6bdda-c44b-494a-a887-538665ecd2c2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2", "tag": "29b83b99-dced-4119-b3f1-cb6ec3d0c385"}]} {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:15:40.901493 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-23c9a14e-c59c-4b2f-b347-1190cd0da8e0 req-c4a6bdda-c44b-494a-a887-538665ecd2c2 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:40.901937 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [req-23c9a14e-c59c-4b2f-b347-1190cd0da8e0 req-c4a6bdda-c44b-494a-a887-538665ecd2c2 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:40.913706 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.compute.server_external_events [req-23c9a14e-c59c-4b2f-b347-1190cd0da8e0 req-c4a6bdda-c44b-494a-a887-538665ecd2c2 service nova] Creating event network-changed:29b83b99-dced-4119-b3f1-cb6ec3d0c385 for instance 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 on hv-cinder-maste Jul 19 11:15:40.918806 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [req-23c9a14e-c59c-4b2f-b347-1190cd0da8e0 req-c4a6bdda-c44b-494a-a887-538665ecd2c2 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036608 Jul 19 11:15:40.920039 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 645/1282] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:15:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:15:40.944895 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-a9abeef2-49e1-4e2e-aeb6-74a9692ad1dc tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "DELETE /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 204 len: 0 microversion: 2.1 time: 0.145360 Jul 19 11:15:40.945475 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 638/1283] 10.222.0.33 () {58 vars in 1238 bytes} [Tue Jul 19 11:15:40 2022] DELETE /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 19 11:15:40.954257 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-37910d31-e0bd-4da2-a9a4-70cb8dc93409 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:40.954946 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-37910d31-e0bd-4da2-a9a4-70cb8dc93409 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:40.961290 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-37910d31-e0bd-4da2-a9a4-70cb8dc93409 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:40.961713 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-37910d31-e0bd-4da2-a9a4-70cb8dc93409 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:40.962463 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-37910d31-e0bd-4da2-a9a4-70cb8dc93409 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:40.962896 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-37910d31-e0bd-4da2-a9a4-70cb8dc93409 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:41.097441 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-37910d31-e0bd-4da2-a9a4-70cb8dc93409 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:15:41.105035 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-37910d31-e0bd-4da2-a9a4-70cb8dc93409 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1580 microversion: 2.1 time: 0.155100 Jul 19 11:15:41.105677 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 646/1284] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:15:40 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1580 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:15:41.899793 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-bd31d8e2-57a3-423f-b530-6707738172a8 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:41.900479 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-bd31d8e2-57a3-423f-b530-6707738172a8 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: 11f9f6fe-f51f-4ac4-862d-93d028914265] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:41.906963 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bd31d8e2-57a3-423f-b530-6707738172a8 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:41.907409 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bd31d8e2-57a3-423f-b530-6707738172a8 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:41.908157 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bd31d8e2-57a3-423f-b530-6707738172a8 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:41.908585 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-bd31d8e2-57a3-423f-b530-6707738172a8 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:42.041325 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-bd31d8e2-57a3-423f-b530-6707738172a8 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e228b9868654a6499bce784a631ec2d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c519c231d95d4e14b2db727ea22d3b50', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:15:42.049140 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-bd31d8e2-57a3-423f-b530-6707738172a8 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "GET /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265" status: 200 len: 1514 microversion: 2.1 time: 0.153568 Jul 19 11:15:42.049703 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 639/1285] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:15:41 2022] GET /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265 => generated 1514 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:15:42.123117 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-0adb542e-f3da-47a8-87bb-8b3feb7a5d42 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:42.123743 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-0adb542e-f3da-47a8-87bb-8b3feb7a5d42 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:42.131130 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0adb542e-f3da-47a8-87bb-8b3feb7a5d42 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:42.131576 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0adb542e-f3da-47a8-87bb-8b3feb7a5d42 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:42.132321 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0adb542e-f3da-47a8-87bb-8b3feb7a5d42 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:42.132792 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-0adb542e-f3da-47a8-87bb-8b3feb7a5d42 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:42.262463 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-0adb542e-f3da-47a8-87bb-8b3feb7a5d42 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:15:42.269921 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-0adb542e-f3da-47a8-87bb-8b3feb7a5d42 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1580 microversion: 2.1 time: 0.150658 Jul 19 11:15:42.270483 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 647/1286] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:15:42 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1580 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 19 11:15:43.066543 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-4a31238e-c75f-4345-9e82-350d2ee39ea6 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:43.067143 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-4a31238e-c75f-4345-9e82-350d2ee39ea6 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: 11f9f6fe-f51f-4ac4-862d-93d028914265] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:43.073570 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4a31238e-c75f-4345-9e82-350d2ee39ea6 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:43.074015 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4a31238e-c75f-4345-9e82-350d2ee39ea6 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:43.074769 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4a31238e-c75f-4345-9e82-350d2ee39ea6 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:43.075201 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-4a31238e-c75f-4345-9e82-350d2ee39ea6 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:43.200297 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-4a31238e-c75f-4345-9e82-350d2ee39ea6 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e228b9868654a6499bce784a631ec2d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c519c231d95d4e14b2db727ea22d3b50', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:15:43.208347 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-4a31238e-c75f-4345-9e82-350d2ee39ea6 tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "GET /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265" status: 200 len: 1514 microversion: 2.1 time: 0.145654 Jul 19 11:15:43.208940 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 640/1287] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:15:43 2022] GET /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265 => generated 1514 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:15:43.286768 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-97e0def5-bcd8-4fc5-bda2-972da0da4d01 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:43.287378 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-97e0def5-bcd8-4fc5-bda2-972da0da4d01 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:43.293806 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-97e0def5-bcd8-4fc5-bda2-972da0da4d01 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:43.294261 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-97e0def5-bcd8-4fc5-bda2-972da0da4d01 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:43.295018 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-97e0def5-bcd8-4fc5-bda2-972da0da4d01 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:43.295458 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-97e0def5-bcd8-4fc5-bda2-972da0da4d01 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:43.423783 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-97e0def5-bcd8-4fc5-bda2-972da0da4d01 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:15:43.431476 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-97e0def5-bcd8-4fc5-bda2-972da0da4d01 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1580 microversion: 2.1 time: 0.148565 Jul 19 11:15:43.432040 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 648/1288] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:15:43 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1580 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:15:44.225614 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-35ef6010-d197-40b7-9ed0-93ed5111a2be tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:44.226244 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-35ef6010-d197-40b7-9ed0-93ed5111a2be tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: 11f9f6fe-f51f-4ac4-862d-93d028914265] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:44.233096 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-35ef6010-d197-40b7-9ed0-93ed5111a2be tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:44.233576 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-35ef6010-d197-40b7-9ed0-93ed5111a2be tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:44.234393 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-35ef6010-d197-40b7-9ed0-93ed5111a2be tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:44.234871 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-35ef6010-d197-40b7-9ed0-93ed5111a2be tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:44.454379 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-47a2d5c4-999e-4ec2-8ace-67c3ced0e9fe tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:44.454967 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-47a2d5c4-999e-4ec2-8ace-67c3ced0e9fe tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:44.462026 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-47a2d5c4-999e-4ec2-8ace-67c3ced0e9fe tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:44.462703 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-47a2d5c4-999e-4ec2-8ace-67c3ced0e9fe tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:44.463698 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-47a2d5c4-999e-4ec2-8ace-67c3ced0e9fe tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:44.463877 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.policy [None req-35ef6010-d197-40b7-9ed0-93ed5111a2be tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e228b9868654a6499bce784a631ec2d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c519c231d95d4e14b2db727ea22d3b50', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117849) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:15:44.464478 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-47a2d5c4-999e-4ec2-8ace-67c3ced0e9fe tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:44.476033 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-35ef6010-d197-40b7-9ed0-93ed5111a2be tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "GET /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265" status: 200 len: 1514 microversion: 2.1 time: 0.254409 Jul 19 11:15:44.477389 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 641/1289] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:15:44 2022] GET /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265 => generated 1514 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:15:44.568313 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-f5193561-b410-45a1-979e-b0261108cd90 req-ced98360-b99a-409a-ac97-c200a6294390 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "11f9f6fe-f51f-4ac4-862d-93d028914265", "name": "network-vif-deleted", "tag": "71fc0996-d492-4438-bd06-b573505fa104"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:15:44.577131 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-f5193561-b410-45a1-979e-b0261108cd90 req-ced98360-b99a-409a-ac97-c200a6294390 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:44.578526 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-f5193561-b410-45a1-979e-b0261108cd90 req-ced98360-b99a-409a-ac97-c200a6294390 service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:44.591662 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-f5193561-b410-45a1-979e-b0261108cd90 req-ced98360-b99a-409a-ac97-c200a6294390 service nova] Creating event network-vif-deleted:71fc0996-d492-4438-bd06-b573505fa104 for instance 11f9f6fe-f51f-4ac4-862d-93d028914265 on hv-cinder-maste Jul 19 11:15:44.596341 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-f5193561-b410-45a1-979e-b0261108cd90 req-ced98360-b99a-409a-ac97-c200a6294390 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.098412 Jul 19 11:15:44.596976 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 642/1290] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:15:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 19 11:15:44.703181 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.policy [None req-47a2d5c4-999e-4ec2-8ace-67c3ced0e9fe tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a217e983effa43f3a4f84221984189c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7109adf88ac4391aee35c150e96e0b1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117848) authorize /opt/stack/nova/nova/policy.py:199}} Jul 19 11:15:44.710813 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-47a2d5c4-999e-4ec2-8ace-67c3ced0e9fe tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 200 len: 1580 microversion: 2.1 time: 0.260902 Jul 19 11:15:44.711393 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 649/1291] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:15:44 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 1580 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 19 11:15:44.812158 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [req-5426aff6-2252-4710-8b9f-73d3f2a864ce req-355718cb-ee38-4973-a1bd-bf057783cf7f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2", "name": "network-vif-deleted", "tag": "29b83b99-dced-4119-b3f1-cb6ec3d0c385"}]} {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 19 11:15:44.820295 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-5426aff6-2252-4710-8b9f-73d3f2a864ce req-355718cb-ee38-4973-a1bd-bf057783cf7f service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:44.820784 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [req-5426aff6-2252-4710-8b9f-73d3f2a864ce req-355718cb-ee38-4973-a1bd-bf057783cf7f service nova] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:44.833802 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.compute.server_external_events [req-5426aff6-2252-4710-8b9f-73d3f2a864ce req-355718cb-ee38-4973-a1bd-bf057783cf7f service nova] Creating event network-vif-deleted:29b83b99-dced-4119-b3f1-cb6ec3d0c385 for instance 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 on hv-cinder-maste Jul 19 11:15:44.838760 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [req-5426aff6-2252-4710-8b9f-73d3f2a864ce req-355718cb-ee38-4973-a1bd-bf057783cf7f service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031140 Jul 19 11:15:44.839335 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 643/1292] 10.222.0.33 () {64 vars in 1317 bytes} [Tue Jul 19 11:15:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 19 11:15:45.493256 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-5a49fb72-8710-40b1-bd82-d921d9c967dd tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:45.493853 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-5a49fb72-8710-40b1-bd82-d921d9c967dd tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] [instance: 11f9f6fe-f51f-4ac4-862d-93d028914265] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:45.500280 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5a49fb72-8710-40b1-bd82-d921d9c967dd tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:45.500748 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5a49fb72-8710-40b1-bd82-d921d9c967dd tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:45.501760 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5a49fb72-8710-40b1-bd82-d921d9c967dd tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:45.502187 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-5a49fb72-8710-40b1-bd82-d921d9c967dd tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:45.515113 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.wsgi [None req-5a49fb72-8710-40b1-bd82-d921d9c967dd tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] HTTP exception thrown: Instance 11f9f6fe-f51f-4ac4-862d-93d028914265 could not be found. Jul 19 11:15:45.516736 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-5a49fb72-8710-40b1-bd82-d921d9c967dd tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Request method failure captured: Jul 19 11:15:45.516736 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: request: GET /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265 HTTP/1.1 Jul 19 11:15:45.516736 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept: application/json Jul 19 11:15:45.516736 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept-Encoding: identity Jul 19 11:15:45.516736 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Connection: close Jul 19 11:15:45.516736 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Length: 0 Jul 19 11:15:45.516736 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Type: application/json Jul 19 11:15:45.516736 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Host: 10.222.0.33 Jul 19 11:15:45.516736 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Openstack-System-Scope: None Jul 19 11:15:45.516736 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: User-Agent: python-urllib3/1.26.9 Jul 19 11:15:45.516736 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Auth-Token: gAAAAABi1pH8lyfocu9X1JPAORymcw6iMWQS5GQILGfG18_1gjd48cM0lKWmDRWFkkyCCVqSEyVedgrrad054F634JGUrerb-ztByhds9xBZFbOJmbOXxJLvmTg4yscuVJodZW0FbAmmPyV6nps5D2e3OfcNKTYlvHVO0srx315_D3X122e6yXo Jul 19 11:15:45.516736 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Id: None Jul 19 11:15:45.516736 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Name: None Jul 19 11:15:45.516736 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Identity-Status: Confirmed Jul 19 11:15:45.516736 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Is-Admin-Project: True Jul 19 11:15:45.516736 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Id: default Jul 19 11:15:45.516736 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Name: Default Jul 19 11:15:45.516736 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Id: c519c231d95d4e14b2db727ea22d3b50 Jul 19 11:15:45.516736 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1651191866 Jul 19 11:15:45.516736 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Role: member,reader Jul 19 11:15:45.516736 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Roles: member,reader Jul 19 11:15:45.518393 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/c519c231d95d4e14b2db727ea22d3b50"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/c519c231d95d4e14b2db727ea22d3b50"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/c519c231d95d4e14b2db727ea22d3b50"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_c519c231d95d4e14b2db727ea22d3b50"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:15:45.518393 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant: tempest-VolumesSnapshotTestJSON-1651191866 Jul 19 11:15:45.518393 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Id: c519c231d95d4e14b2db727ea22d3b50 Jul 19 11:15:45.518393 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1651191866 Jul 19 11:15:45.518393 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User: tempest-VolumesSnapshotTestJSON-1651191866-project Jul 19 11:15:45.518393 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Id: default Jul 19 11:15:45.518393 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Name: Default Jul 19 11:15:45.518393 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Id: 8e228b9868654a6499bce784a631ec2d Jul 19 11:15:45.518393 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Name: tempest-VolumesSnapshotTestJSON-1651191866-project Jul 19 11:15:45.518393 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: method: > Jul 19 11:15:45.518393 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: exception: Instance 11f9f6fe-f51f-4ac4-862d-93d028914265 could not be found. Jul 19 11:15:45.518393 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: action_args: {'id': '11f9f6fe-f51f-4ac4-862d-93d028914265'} Jul 19 11:15:45.519291 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:15:45.519291 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:15:45.519291 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:15:45.519291 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:15:45.519291 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:15:45.519291 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:15:45.519291 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:15:45.519291 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:15:45.519291 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:15:45.519291 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:15:45.519291 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 11f9f6fe-f51f-4ac4-862d-93d028914265 could not be found. Jul 19 11:15:45.519291 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:15:45.519291 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:15:45.519291 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:15:45.519291 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:15:45.519291 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:15:45.519291 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:15:45.519291 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:15:45.519291 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:15:45.519291 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:15:45.520643 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:15:45.520643 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 19 11:15:45.520643 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 19 11:15:45.520643 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:15:45.520643 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:15:45.520643 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:15:45.520643 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:15:45.520643 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 11f9f6fe-f51f-4ac4-862d-93d028914265 could not be found. Jul 19 11:15:45.520643 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:15:45.520643 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:15:45.520643 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:15:45.520643 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:15:45.520643 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:15:45.520643 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:15:45.520643 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:15:45.520643 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:15:45.520643 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 11f9f6fe-f51f-4ac4-862d-93d028914265 could not be found. Jul 19 11:15:45.520643 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:15:45.521906 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-5a49fb72-8710-40b1-bd82-d921d9c967dd tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] Returning 404 to user: Instance 11f9f6fe-f51f-4ac4-862d-93d028914265 could not be found. {{(pid=117848) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:15:45.521906 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-5a49fb72-8710-40b1-bd82-d921d9c967dd tempest-VolumesSnapshotTestJSON-1651191866 tempest-VolumesSnapshotTestJSON-1651191866-project] 10.222.0.33 "GET /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265" status: 404 len: 111 microversion: 2.1 time: 0.028734 Jul 19 11:15:45.521906 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 650/1293] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:15:45 2022] GET /compute/v2.1/servers/11f9f6fe-f51f-4ac4-862d-93d028914265 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:15:45.728460 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-35be6c48-b4d2-4b62-926d-e79649b47912 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:45.729178 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-35be6c48-b4d2-4b62-926d-e79649b47912 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:45.740179 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-35be6c48-b4d2-4b62-926d-e79649b47912 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:45.740179 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-35be6c48-b4d2-4b62-926d-e79649b47912 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:45.740179 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-35be6c48-b4d2-4b62-926d-e79649b47912 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:45.740179 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-35be6c48-b4d2-4b62-926d-e79649b47912 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:45.759214 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.wsgi [None req-35be6c48-b4d2-4b62-926d-e79649b47912 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] HTTP exception thrown: Instance 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 could not be found. Jul 19 11:15:45.761144 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-35be6c48-b4d2-4b62-926d-e79649b47912 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Request method failure captured: Jul 19 11:15:45.761144 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: request: GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 HTTP/1.1 Jul 19 11:15:45.761144 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept: application/json Jul 19 11:15:45.761144 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept-Encoding: identity Jul 19 11:15:45.761144 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Connection: close Jul 19 11:15:45.761144 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Length: 0 Jul 19 11:15:45.761144 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Type: application/json Jul 19 11:15:45.761144 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Host: 10.222.0.33 Jul 19 11:15:45.761144 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Openstack-System-Scope: None Jul 19 11:15:45.761144 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: User-Agent: python-urllib3/1.26.9 Jul 19 11:15:45.761144 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Auth-Token: gAAAAABi1pCKSZ-4SJ5s_ItAgtEK2gvn4DyrOW7x3YHwCqO5aRGl0Sxm4ZZRq5InRFnucVfPkLXEpzi5hBz6xHVKVr9nFez8ydrjVvbO97mMRi8k_yB3xxgwAy6BC0nPq6dyYILo3FmB20Si0ydiOAi4CYj6_z7hgsm3RTs6jezifIIUuhEeGdE Jul 19 11:15:45.761144 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Id: None Jul 19 11:15:45.761144 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Name: None Jul 19 11:15:45.761144 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Identity-Status: Confirmed Jul 19 11:15:45.761144 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Is-Admin-Project: True Jul 19 11:15:45.761144 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Id: default Jul 19 11:15:45.761144 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Name: Default Jul 19 11:15:45.761144 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Id: a7109adf88ac4391aee35c150e96e0b1 Jul 19 11:15:45.761144 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Name: tempest-TestVolumeBootPattern-1429943467 Jul 19 11:15:45.761144 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Role: member,reader Jul 19 11:15:45.761144 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Roles: member,reader Jul 19 11:15:45.762881 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7109adf88ac4391aee35c150e96e0b1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/a7109adf88ac4391aee35c150e96e0b1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7109adf88ac4391aee35c150e96e0b1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_a7109adf88ac4391aee35c150e96e0b1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:15:45.762881 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant: tempest-TestVolumeBootPattern-1429943467 Jul 19 11:15:45.762881 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Id: a7109adf88ac4391aee35c150e96e0b1 Jul 19 11:15:45.762881 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Name: tempest-TestVolumeBootPattern-1429943467 Jul 19 11:15:45.762881 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User: tempest-TestVolumeBootPattern-1429943467-project Jul 19 11:15:45.762881 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Id: default Jul 19 11:15:45.762881 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Name: Default Jul 19 11:15:45.762881 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Id: a217e983effa43f3a4f84221984189c5 Jul 19 11:15:45.762881 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Name: tempest-TestVolumeBootPattern-1429943467-project Jul 19 11:15:45.762881 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: method: > Jul 19 11:15:45.762881 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: exception: Instance 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 could not be found. Jul 19 11:15:45.762881 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: action_args: {'id': '3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2'} Jul 19 11:15:45.763834 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:15:45.763834 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:15:45.763834 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:15:45.763834 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:15:45.763834 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:15:45.763834 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:15:45.763834 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:15:45.763834 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:15:45.763834 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:15:45.763834 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:15:45.763834 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 could not be found. Jul 19 11:15:45.763834 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:15:45.763834 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:15:45.763834 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:15:45.763834 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:15:45.763834 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:15:45.763834 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:15:45.763834 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:15:45.763834 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:15:45.763834 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:15:45.765921 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:15:45.765921 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 19 11:15:45.765921 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 19 11:15:45.765921 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:15:45.765921 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:15:45.765921 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:15:45.765921 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:15:45.765921 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 could not be found. Jul 19 11:15:45.765921 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:15:45.765921 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:15:45.765921 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:15:45.765921 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:15:45.765921 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:15:45.765921 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:15:45.765921 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:15:45.765921 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:15:45.765921 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 could not be found. Jul 19 11:15:45.765921 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:15:45.767306 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-35be6c48-b4d2-4b62-926d-e79649b47912 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Returning 404 to user: Instance 3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 could not be found. {{(pid=117849) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:15:45.767306 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-35be6c48-b4d2-4b62-926d-e79649b47912 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2" status: 404 len: 111 microversion: 2.1 time: 0.038920 Jul 19 11:15:45.772852 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 644/1294] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:15:45 2022] GET /compute/v2.1/servers/3bb24318-baf4-40e3-b2d2-f8f3cd36e1e2 => generated 111 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:15:46.217561 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-16a56b43-ce7e-43ed-bcc6-f87973627988 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:46.218102 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.compute.api [None req-16a56b43-ce7e-43ed-bcc6-f87973627988 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 12cd8c55-b30f-40ee-8cf5-7c974b8caafb] Fetching instance by UUID {{(pid=117848) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:46.224239 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-16a56b43-ce7e-43ed-bcc6-f87973627988 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:46.224736 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-16a56b43-ce7e-43ed-bcc6-f87973627988 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:46.225516 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-16a56b43-ce7e-43ed-bcc6-f87973627988 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:46.225952 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG oslo_concurrency.lockutils [None req-16a56b43-ce7e-43ed-bcc6-f87973627988 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117848) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:46.239257 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.wsgi [None req-16a56b43-ce7e-43ed-bcc6-f87973627988 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] HTTP exception thrown: Instance 12cd8c55-b30f-40ee-8cf5-7c974b8caafb could not be found. Jul 19 11:15:46.240822 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-16a56b43-ce7e-43ed-bcc6-f87973627988 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Request method failure captured: Jul 19 11:15:46.240822 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: request: DELETE /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb HTTP/1.1 Jul 19 11:15:46.240822 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept: application/json Jul 19 11:15:46.240822 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Accept-Encoding: identity Jul 19 11:15:46.240822 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Connection: close Jul 19 11:15:46.240822 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Length: 0 Jul 19 11:15:46.240822 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Content-Type: application/json Jul 19 11:15:46.240822 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Host: 10.222.0.33 Jul 19 11:15:46.240822 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Openstack-System-Scope: None Jul 19 11:15:46.240822 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: User-Agent: python-urllib3/1.26.9 Jul 19 11:15:46.240822 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Auth-Token: gAAAAABi1pCKSZ-4SJ5s_ItAgtEK2gvn4DyrOW7x3YHwCqO5aRGl0Sxm4ZZRq5InRFnucVfPkLXEpzi5hBz6xHVKVr9nFez8ydrjVvbO97mMRi8k_yB3xxgwAy6BC0nPq6dyYILo3FmB20Si0ydiOAi4CYj6_z7hgsm3RTs6jezifIIUuhEeGdE Jul 19 11:15:46.240822 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Id: None Jul 19 11:15:46.240822 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Domain-Name: None Jul 19 11:15:46.240822 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Identity-Status: Confirmed Jul 19 11:15:46.240822 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Is-Admin-Project: True Jul 19 11:15:46.240822 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Id: default Jul 19 11:15:46.240822 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Domain-Name: Default Jul 19 11:15:46.240822 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Id: a7109adf88ac4391aee35c150e96e0b1 Jul 19 11:15:46.240822 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Project-Name: tempest-TestVolumeBootPattern-1429943467 Jul 19 11:15:46.240822 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Role: member,reader Jul 19 11:15:46.240822 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Roles: member,reader Jul 19 11:15:46.242471 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7109adf88ac4391aee35c150e96e0b1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/a7109adf88ac4391aee35c150e96e0b1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7109adf88ac4391aee35c150e96e0b1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_a7109adf88ac4391aee35c150e96e0b1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:15:46.242471 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant: tempest-TestVolumeBootPattern-1429943467 Jul 19 11:15:46.242471 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Id: a7109adf88ac4391aee35c150e96e0b1 Jul 19 11:15:46.242471 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-Tenant-Name: tempest-TestVolumeBootPattern-1429943467 Jul 19 11:15:46.242471 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User: tempest-TestVolumeBootPattern-1429943467-project Jul 19 11:15:46.242471 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Id: default Jul 19 11:15:46.242471 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Domain-Name: Default Jul 19 11:15:46.242471 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Id: a217e983effa43f3a4f84221984189c5 Jul 19 11:15:46.242471 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: X-User-Name: tempest-TestVolumeBootPattern-1429943467-project Jul 19 11:15:46.242471 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: method: > Jul 19 11:15:46.242471 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: exception: Instance 12cd8c55-b30f-40ee-8cf5-7c974b8caafb could not be found. Jul 19 11:15:46.242471 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: action_args: {'id': '12cd8c55-b30f-40ee-8cf5-7c974b8caafb'} Jul 19 11:15:46.243448 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:15:46.243448 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:15:46.243448 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:15:46.243448 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:15:46.243448 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:15:46.243448 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:15:46.243448 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:15:46.243448 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:15:46.243448 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:15:46.243448 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:15:46.243448 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 12cd8c55-b30f-40ee-8cf5-7c974b8caafb could not be found. Jul 19 11:15:46.243448 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:15:46.243448 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:15:46.243448 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:15:46.243448 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:15:46.243448 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:15:46.243448 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:15:46.243448 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:15:46.243448 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:15:46.243448 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:15:46.245131 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:15:46.245131 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 19 11:15:46.245131 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 19 11:15:46.245131 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 19 11:15:46.245131 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 19 11:15:46.245131 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:15:46.245131 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:15:46.245131 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:15:46.245131 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:15:46.245131 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 12cd8c55-b30f-40ee-8cf5-7c974b8caafb could not be found. Jul 19 11:15:46.245131 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:15:46.245131 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:15:46.245131 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:15:46.245131 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:15:46.245131 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:15:46.245131 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:15:46.245131 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:15:46.245131 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:15:46.245131 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 12cd8c55-b30f-40ee-8cf5-7c974b8caafb could not be found. Jul 19 11:15:46.245131 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: ERROR nova.api.openstack.wsgi Jul 19 11:15:46.246995 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-16a56b43-ce7e-43ed-bcc6-f87973627988 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Returning 404 to user: Instance 12cd8c55-b30f-40ee-8cf5-7c974b8caafb could not be found. {{(pid=117848) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:15:46.246995 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-16a56b43-ce7e-43ed-bcc6-f87973627988 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "DELETE /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb" status: 404 len: 111 microversion: 2.1 time: 0.028839 Jul 19 11:15:46.246995 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 651/1295] 10.222.0.33 () {58 vars in 1238 bytes} [Tue Jul 19 11:15:46 2022] DELETE /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:15:46.252457 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-5255df16-f3fc-46e0-b093-748d4f3050e9 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '>' {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:46.253090 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.compute.api [None req-5255df16-f3fc-46e0-b093-748d4f3050e9 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] [instance: 12cd8c55-b30f-40ee-8cf5-7c974b8caafb] Fetching instance by UUID {{(pid=117849) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 19 11:15:46.259330 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5255df16-f3fc-46e0-b093-748d4f3050e9 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:46.259806 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5255df16-f3fc-46e0-b093-748d4f3050e9 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:46.260558 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5255df16-f3fc-46e0-b093-748d4f3050e9 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 19 11:15:46.261017 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG oslo_concurrency.lockutils [None req-5255df16-f3fc-46e0-b093-748d4f3050e9 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Lock "8c617d32-2606-4a09-a34c-e793c0a350a1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117849) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 19 11:15:46.274957 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.wsgi [None req-5255df16-f3fc-46e0-b093-748d4f3050e9 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] HTTP exception thrown: Instance 12cd8c55-b30f-40ee-8cf5-7c974b8caafb could not be found. Jul 19 11:15:46.276437 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-5255df16-f3fc-46e0-b093-748d4f3050e9 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Request method failure captured: Jul 19 11:15:46.276437 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: request: GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb HTTP/1.1 Jul 19 11:15:46.276437 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept: application/json Jul 19 11:15:46.276437 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Accept-Encoding: identity Jul 19 11:15:46.276437 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Connection: close Jul 19 11:15:46.276437 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Length: 0 Jul 19 11:15:46.276437 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Content-Type: application/json Jul 19 11:15:46.276437 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Host: 10.222.0.33 Jul 19 11:15:46.276437 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Openstack-System-Scope: None Jul 19 11:15:46.276437 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: User-Agent: python-urllib3/1.26.9 Jul 19 11:15:46.276437 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Auth-Token: gAAAAABi1pCKSZ-4SJ5s_ItAgtEK2gvn4DyrOW7x3YHwCqO5aRGl0Sxm4ZZRq5InRFnucVfPkLXEpzi5hBz6xHVKVr9nFez8ydrjVvbO97mMRi8k_yB3xxgwAy6BC0nPq6dyYILo3FmB20Si0ydiOAi4CYj6_z7hgsm3RTs6jezifIIUuhEeGdE Jul 19 11:15:46.276437 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Id: None Jul 19 11:15:46.276437 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Domain-Name: None Jul 19 11:15:46.276437 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Identity-Status: Confirmed Jul 19 11:15:46.276437 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Is-Admin-Project: True Jul 19 11:15:46.276437 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Id: default Jul 19 11:15:46.276437 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Domain-Name: Default Jul 19 11:15:46.276437 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Id: a7109adf88ac4391aee35c150e96e0b1 Jul 19 11:15:46.276437 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Project-Name: tempest-TestVolumeBootPattern-1429943467 Jul 19 11:15:46.276437 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Role: member,reader Jul 19 11:15:46.276437 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Roles: member,reader Jul 19 11:15:46.278187 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7109adf88ac4391aee35c150e96e0b1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/a7109adf88ac4391aee35c150e96e0b1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7109adf88ac4391aee35c150e96e0b1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_a7109adf88ac4391aee35c150e96e0b1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}] Jul 19 11:15:46.278187 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant: tempest-TestVolumeBootPattern-1429943467 Jul 19 11:15:46.278187 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Id: a7109adf88ac4391aee35c150e96e0b1 Jul 19 11:15:46.278187 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-Tenant-Name: tempest-TestVolumeBootPattern-1429943467 Jul 19 11:15:46.278187 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User: tempest-TestVolumeBootPattern-1429943467-project Jul 19 11:15:46.278187 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Id: default Jul 19 11:15:46.278187 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Domain-Name: Default Jul 19 11:15:46.278187 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Id: a217e983effa43f3a4f84221984189c5 Jul 19 11:15:46.278187 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: X-User-Name: tempest-TestVolumeBootPattern-1429943467-project Jul 19 11:15:46.278187 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: method: > Jul 19 11:15:46.278187 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: exception: Instance 12cd8c55-b30f-40ee-8cf5-7c974b8caafb could not be found. Jul 19 11:15:46.278187 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: action_args: {'id': '12cd8c55-b30f-40ee-8cf5-7c974b8caafb'} Jul 19 11:15:46.279177 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: {{(pid=117849) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 19 11:15:46.279177 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:15:46.279177 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 19 11:15:46.279177 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 19 11:15:46.279177 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 19 11:15:46.279177 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 19 11:15:46.279177 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 19 11:15:46.279177 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 19 11:15:46.279177 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 19 11:15:46.279177 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 19 11:15:46.279177 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 12cd8c55-b30f-40ee-8cf5-7c974b8caafb could not be found. Jul 19 11:15:46.279177 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:15:46.279177 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:15:46.279177 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:15:46.279177 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:15:46.279177 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:15:46.279177 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:15:46.279177 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 19 11:15:46.279177 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 19 11:15:46.279177 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 19 11:15:46.280944 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 19 11:15:46.280944 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 19 11:15:46.280944 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 19 11:15:46.280944 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 19 11:15:46.280944 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 19 11:15:46.280944 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 19 11:15:46.280944 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 19 11:15:46.280944 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 12cd8c55-b30f-40ee-8cf5-7c974b8caafb could not be found. Jul 19 11:15:46.280944 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:15:46.280944 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 19 11:15:46.280944 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:15:46.280944 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 19 11:15:46.280944 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 19 11:15:46.280944 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 19 11:15:46.280944 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 19 11:15:46.280944 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 19 11:15:46.280944 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 12cd8c55-b30f-40ee-8cf5-7c974b8caafb could not be found. Jul 19 11:15:46.280944 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: ERROR nova.api.openstack.wsgi Jul 19 11:15:46.282377 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: DEBUG nova.api.openstack.wsgi [None req-5255df16-f3fc-46e0-b093-748d4f3050e9 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Returning 404 to user: Instance 12cd8c55-b30f-40ee-8cf5-7c974b8caafb could not be found. {{(pid=117849) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 19 11:15:46.282377 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO nova.api.openstack.requestlog [None req-5255df16-f3fc-46e0-b093-748d4f3050e9 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb" status: 404 len: 111 microversion: 2.1 time: 0.030395 Jul 19 11:15:46.282377 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: [pid: 117849|app: 0|req: 645/1296] 10.222.0.33 () {58 vars in 1235 bytes} [Tue Jul 19 11:15:46 2022] GET /compute/v2.1/servers/12cd8c55-b30f-40ee-8cf5-7c974b8caafb => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 19 11:15:47.559886 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: DEBUG nova.api.openstack.wsgi [None req-80368a26-9aab-4358-821c-190993aa6eb1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] Calling method '.version_select at 0x7f04aa7975e0>' {{(pid=117848) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 19 11:15:47.579124 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO nova.api.openstack.requestlog [None req-80368a26-9aab-4358-821c-190993aa6eb1 tempest-TestVolumeBootPattern-1429943467 tempest-TestVolumeBootPattern-1429943467-project] 10.222.0.33 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1205270003" status: 202 len: 0 microversion: 2.1 time: 0.023066 Jul 19 11:15:47.579715 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: [pid: 117848|app: 0|req: 652/1297] 10.222.0.33 () {58 vars in 1262 bytes} [Tue Jul 19 11:15:47 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1205270003 => generated 0 bytes in 24 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 19 11:18:00.029174 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 19 11:18:00.033725 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Tue Jul 19 11:18:00 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 19 11:18:01.232663 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 19 11:18:01.233204 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Tue Jul 19 11:18:01 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 19 11:18:16.020368 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 19 11:18:16.020916 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Tue Jul 19 11:18:16 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 19 11:18:21.013638 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 19 11:18:21.014075 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Tue Jul 19 11:18:21 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 19 11:19:37.524036 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 19 11:19:37.524632 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Tue Jul 19 11:19:37 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 19 11:21:20.038728 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 19 11:21:20.039484 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Tue Jul 19 11:21:20 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 19 11:21:20.792420 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 19 11:21:20.793046 dv-cinder-master-3-iscsi devstack@n-api.service[117848]: Tue Jul 19 11:21:20 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 19 11:21:21.240611 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 19 11:21:21.241079 dv-cinder-master-3-iscsi devstack@n-api.service[117849]: Tue Jul 19 11:21:21 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!!